Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260808 13:34:40.911059 4776 test_util.cc:268] Using random seed: 786883144
2026-08-08T13:34:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:34:40Z Disabled control of system clock
I20260808 13:34:40.970844 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.62:34069
--webserver_interface=127.4.170.62
--webserver_port=0
--builtin_ntp_servers=127.4.170.20:42231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260808 13:34:41.281672 4791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:34:41.282362 4791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:34:41.282806 4791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:34:41.313189 4791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:34:41.313489 4791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:34:41.313694 4791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:34:41.313894 4791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:34:41.348903 4791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.62:34069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.170.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:34:41.350322 4791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:34:41.352183 4791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:34:41.372164 4810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:34:41.372193 4807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:34:41.372275 4808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:34:41.373948 4791 server_base.cc:1034] running on GCE node
I20260808 13:34:42.639529 4791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:34:42.642480 4791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:34:42.643941 4791 hybrid_clock.cc:648] HybridClock initialized: now 1786196082643900 us; error 126 us; skew 500 ppm
I20260808 13:34:42.644824 4791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:34:42.652189 4791 webserver.cc:460] Webserver started at http://127.4.170.62:36097/ using document root <none> and password file <none>
I20260808 13:34:42.653307 4791 fs_manager.cc:359] Metadata directory not provided
I20260808 13:34:42.653571 4791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:34:42.654094 4791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:34:42.659451 4791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/data/instance:
uuid: "3e672d7386144b299688abea9c64854c"
format_stamp: "Formatted at 2026-08-08 13:34:42 on dist-test-slave-twwt"
I20260808 13:34:42.660586 4791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/wal/instance:
uuid: "3e672d7386144b299688abea9c64854c"
format_stamp: "Formatted at 2026-08-08 13:34:42 on dist-test-slave-twwt"
I20260808 13:34:42.668694 4791 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260808 13:34:42.674355 4818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:34:42.675582 4791 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260808 13:34:42.675938 4791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/wal
uuid: "3e672d7386144b299688abea9c64854c"
format_stamp: "Formatted at 2026-08-08 13:34:42 on dist-test-slave-twwt"
I20260808 13:34:42.676295 4791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:34:42.721985 4791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:34:42.723651 4791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:34:42.724174 4791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:34:42.875981 4791 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.62:34069
I20260808 13:34:42.876176 4899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.62:34069 every 8 connection(s)
I20260808 13:34:42.879169 4791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/data/info.pb
I20260808 13:34:42.885104 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 4791
I20260808 13:34:42.885864 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-0/wal/instance
I20260808 13:34:42.890964 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.61:46555
--webserver_interface=127.4.170.61
--webserver_port=0
--builtin_ntp_servers=127.4.170.20:42231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260808 13:34:42.891165 4900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:34:42.915841 4900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:34:42.967082 4900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } has no permanent_uuid. Determining permanent_uuid...
W20260808 13:34:42.971758 4820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.61:46555: connect: Connection refused (error 111)
W20260808 13:34:42.983404 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:46555: Network error: Client connection negotiation failed: client connection to 127.4.170.61:46555: connect: Connection refused (error 111)
I20260808 13:34:43.033165 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } attempt: 1
W20260808 13:34:43.040489 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:46555: Network error: Client connection negotiation failed: client connection to 127.4.170.61:46555: connect: Connection refused (error 111)
I20260808 13:34:43.109148 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } attempt: 2
W20260808 13:34:43.115422 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:46555: Network error: Client connection negotiation failed: client connection to 127.4.170.61:46555: connect: Connection refused (error 111)
I20260808 13:34:43.206923 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } attempt: 3
W20260808 13:34:43.213169 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:46555: Network error: Client connection negotiation failed: client connection to 127.4.170.61:46555: connect: Connection refused (error 111)
I20260808 13:34:43.356709 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } attempt: 4
W20260808 13:34:43.363950 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:46555: Network error: Client connection negotiation failed: client connection to 127.4.170.61:46555: connect: Connection refused (error 111)
W20260808 13:34:43.406999 4904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:34:43.407665 4904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:34:43.408231 4904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:34:43.440697 4904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:34:43.441146 4904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:34:43.441424 4904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:34:43.441684 4904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:34:43.479584 4904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.61:46555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.170.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:34:43.480967 4904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:34:43.482736 4904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:34:43.506279 4922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:34:43.663482 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } attempt: 5
W20260808 13:34:43.674288 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:46555: Network error: Client connection negotiation failed: client connection to 127.4.170.61:46555: connect: Connection refused (error 111)
I20260808 13:34:44.222106 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } attempt: 6
W20260808 13:34:44.255097 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:46555: Network error: Client connection negotiation failed: client connection to 127.4.170.61:46555: connect: Connection refused (error 111)
W20260808 13:34:43.506531 4921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:34:44.915228 4923 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1406 milliseconds
W20260808 13:34:44.916544 4924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:34:44.918659 4904 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 1.414s user 0.508s sys 0.889s
W20260808 13:34:44.918967 4904 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 1.414s user 0.508s sys 0.893s
I20260808 13:34:44.919250 4904 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260808 13:34:44.920500 4904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:34:44.924482 4904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:34:44.925948 4904 hybrid_clock.cc:648] HybridClock initialized: now 1786196084925891 us; error 65 us; skew 500 ppm
I20260808 13:34:44.927044 4904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:34:44.938972 4904 webserver.cc:460] Webserver started at http://127.4.170.61:37929/ using document root <none> and password file <none>
I20260808 13:34:44.940069 4904 fs_manager.cc:359] Metadata directory not provided
I20260808 13:34:44.940426 4904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:34:44.940945 4904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:34:44.947525 4904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/data/instance:
uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7"
format_stamp: "Formatted at 2026-08-08 13:34:44 on dist-test-slave-twwt"
I20260808 13:34:44.948933 4904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/wal/instance:
uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7"
format_stamp: "Formatted at 2026-08-08 13:34:44 on dist-test-slave-twwt"
I20260808 13:34:44.960827 4904 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.006s sys 0.006s
I20260808 13:34:44.970359 4939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:34:44.973160 4904 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.003s
I20260808 13:34:44.973644 4904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/wal
uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7"
format_stamp: "Formatted at 2026-08-08 13:34:44 on dist-test-slave-twwt"
I20260808 13:34:44.974037 4904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:34:45.043473 4904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:34:45.045593 4904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:34:45.046177 4904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:34:45.254115 4904 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.61:46555
I20260808 13:34:45.254456 5014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.61:46555 every 8 connection(s)
I20260808 13:34:45.257584 4904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/data/info.pb
I20260808 13:34:45.260871 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 4904
I20260808 13:34:45.261703 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-1/wal/instance
I20260808 13:34:45.266845 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.60:37135
--webserver_interface=127.4.170.60
--webserver_port=0
--builtin_ntp_servers=127.4.170.20:42231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260808 13:34:45.270967 5018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:34:45.291887 5018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:34:45.315768 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } attempt: 7
I20260808 13:34:45.342739 5018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:34:45.370596 4900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:34:45.378209 5018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } has no permanent_uuid. Determining permanent_uuid...
W20260808 13:34:45.383191 4944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
W20260808 13:34:45.383721 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
W20260808 13:34:45.395663 5018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:34:45.442235 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 1
I20260808 13:34:45.445174 5018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 1
W20260808 13:34:45.449968 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
W20260808 13:34:45.453680 5018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:34:45.522204 5018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 2
W20260808 13:34:45.528362 5018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:34:45.531523 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 2
W20260808 13:34:45.537528 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:34:45.619927 5018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 3
I20260808 13:34:45.623097 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 3
W20260808 13:34:45.626600 5018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
W20260808 13:34:45.629755 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:34:45.770304 5018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 4
I20260808 13:34:45.770387 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 4
W20260808 13:34:45.779101 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
W20260808 13:34:45.780833 5018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
W20260808 13:34:45.825783 5020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:34:45.826431 5020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:34:45.826871 5020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:34:45.859566 5020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:34:45.859866 5020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:34:45.860069 5020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:34:45.860260 5020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:34:45.899847 5020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.60:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.170.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:34:45.901063 5020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:34:45.902726 5020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:34:45.934897 5037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:34:46.062682 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 5
W20260808 13:34:46.070740 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:34:46.080317 5018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 5
W20260808 13:34:46.088891 5018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:34:46.623545 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 6
I20260808 13:34:46.636583 5018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 6
W20260808 13:34:46.640990 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
W20260808 13:34:46.654585 5018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
W20260808 13:34:47.329521 5036 debug-util.cc:398] Leaking SignalData structure 0x7b0800014860 after lost signal to thread 5020
W20260808 13:34:47.384783 5020 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 1.456s user 0.477s sys 0.971s
W20260808 13:34:47.385131 5020 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 1.456s user 0.477s sys 0.971s
W20260808 13:34:45.935343 5039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:34:47.385811 5040 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260808 13:34:47.388295 5020 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
W20260808 13:34:47.389041 5041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:34:47.391459 5020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:34:47.395318 5020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:34:47.396692 5020 hybrid_clock.cc:648] HybridClock initialized: now 1786196087396650 us; error 45 us; skew 500 ppm
I20260808 13:34:47.397747 5020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:34:47.417858 5020 webserver.cc:460] Webserver started at http://127.4.170.60:45565/ using document root <none> and password file <none>
I20260808 13:34:47.419081 5020 fs_manager.cc:359] Metadata directory not provided
I20260808 13:34:47.419379 5020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:34:47.419866 5020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:34:47.425995 5020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data/instance:
uuid: "87e8b888041b4135af55792050defcb5"
format_stamp: "Formatted at 2026-08-08 13:34:47 on dist-test-slave-twwt"
I20260808 13:34:47.427407 5020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal/instance:
uuid: "87e8b888041b4135af55792050defcb5"
format_stamp: "Formatted at 2026-08-08 13:34:47 on dist-test-slave-twwt"
I20260808 13:34:47.439345 5020 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.005s
I20260808 13:34:47.451097 5056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:34:47.452517 5020 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.006s
I20260808 13:34:47.452993 5020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal
uuid: "87e8b888041b4135af55792050defcb5"
format_stamp: "Formatted at 2026-08-08 13:34:47 on dist-test-slave-twwt"
I20260808 13:34:47.453392 5020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:34:47.513978 5020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:34:47.516881 5020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:34:47.517638 5020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:34:47.674726 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 7
W20260808 13:34:47.693035 4900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:34:47.715293 5018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 7
W20260808 13:34:47.733788 5018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:37135: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:34:47.784308 5020 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.60:37135
I20260808 13:34:47.784825 5142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.60:37135 every 8 connection(s)
I20260808 13:34:47.788542 5020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
I20260808 13:34:47.793452 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 5020
I20260808 13:34:47.794164 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal/instance
I20260808 13:34:47.800266 5144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:34:47.846746 5144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:34:47.901839 5144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:34:47.924418 5144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:34:47.960882 5144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Bootstrap starting.
I20260808 13:34:47.974778 5144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Neither blocks nor log segments found. Creating new log.
I20260808 13:34:47.978400 5144 log.cc:824] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Log is configured to *not* fsync() on all Append() calls
I20260808 13:34:47.988714 5144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: No bootstrap required, opened a new log
I20260808 13:34:48.034241 5144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:48.035444 5144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:34:48.035894 5144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e8b888041b4135af55792050defcb5, State: Initialized, Role: FOLLOWER
I20260808 13:34:48.039067 5144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:48.045262 5156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:48.047199 5156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:48.047534 5144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 13:34:48.121191 5170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:34:48.121701 5170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:34:48.108908 5020 master_runner.cc:186] Error getting master registration for 127.4.170.62:34069: OK, instance_id { permanent_uuid: "3e672d7386144b299688abea9c64854c" instance_seqno: 1786196082820297 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:34:48.156596 5020 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:34:48.227106 5156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:34:48.227674 5156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:48.230515 5156 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
W20260808 13:34:48.282369 5067 tablet.cc:2369] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:34:48.804229 5061 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.170.61:46555 timed out after 0.573s (SENT)
W20260808 13:34:49.128413 5170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:34:49.128825 5170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:34:49.383481 5061 leader_election.cc:334] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555): Timed out: RequestConsensusVote RPC to 127.4.170.61:46555 timed out after 0.573s (SENT)
W20260808 13:34:49.384821 5058 leader_election.cc:334] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 3e672d7386144b299688abea9c64854c (127.4.170.62:34069): Timed out: RequestConsensusVote RPC to 127.4.170.62:34069 timed out after 0.573s (SENT)
I20260808 13:34:49.385342 5058 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:34:49.389607 5175 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:34:49.754745 4900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 8
I20260808 13:34:49.817904 4900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Bootstrap starting.
I20260808 13:34:49.824544 5018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } attempt: 8
I20260808 13:34:49.847016 4900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Neither blocks nor log segments found. Creating new log.
I20260808 13:34:49.863134 4900 log.cc:824] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Log is configured to *not* fsync() on all Append() calls
I20260808 13:34:49.891412 4900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: No bootstrap required, opened a new log
I20260808 13:34:49.905757 5018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: Bootstrap starting.
I20260808 13:34:49.921293 5018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: Neither blocks nor log segments found. Creating new log.
I20260808 13:34:49.926218 5018 log.cc:824] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: Log is configured to *not* fsync() on all Append() calls
I20260808 13:34:49.965924 4900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:49.967347 4900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:34:49.967909 4900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e672d7386144b299688abea9c64854c, State: Initialized, Role: FOLLOWER
I20260808 13:34:49.969192 5018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: No bootstrap required, opened a new log
I20260808 13:34:49.969519 4900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:49.979984 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:49.981223 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:49.986377 4900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:34:49.987388 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
W20260808 13:34:49.992362 4827 tablet.cc:2369] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:34:49.988260 4856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
W20260808 13:34:50.004161 4856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49784 (request call id 3) took 1193 ms (client timeout 573 ms). Trace:
W20260808 13:34:50.004923 4856 rpcz_store.cc:269] 0808 13:34:48.810887 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:34:48.811077 (+ 190us) service_pool.cc:227] Handling call
0808 13:34:50.004001 (+1192924us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1209728}
I20260808 13:34:50.007026 4864 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e8b888041b4135af55792050defcb5 in term 0.
W20260808 13:34:50.009382 4864 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49784 (request call id 2) took 1778 ms (client timeout 573 ms). Trace:
W20260808 13:34:50.010105 4864 rpcz_store.cc:269] 0808 13:34:48.231051 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:34:48.231190 (+ 139us) service_pool.cc:227] Handling call
0808 13:34:50.009281 (+1778091us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:34:50.042518 5018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:50.044059 5018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:34:50.044631 5018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63c875fc5bcd41a9b1bcf3bd4ae321d7, State: Initialized, Role: FOLLOWER
I20260808 13:34:50.046259 5018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:50.068148 5198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:50.071859 5198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:50.073465 5018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:34:50.074350 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:50.076499 4980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e8b888041b4135af55792050defcb5 in term 0.
W20260808 13:34:50.077487 4980 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60308 (request call id 1) took 1845 ms (client timeout 573 ms). Trace:
I20260808 13:34:50.079293 4978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:50.080506 4978 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e8b888041b4135af55792050defcb5 in term 0.
W20260808 13:34:50.081972 4978 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60308 (request call id 2) took 1272 ms (client timeout 573 ms). Trace:
W20260808 13:34:50.082885 4978 rpcz_store.cc:269] 0808 13:34:48.810015 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:34:48.810179 (+ 164us) service_pool.cc:227] Handling call
0808 13:34:50.081887 (+1271708us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260808 13:34:50.078089 4980 rpcz_store.cc:269] 0808 13:34:48.232011 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:34:48.232338 (+ 327us) service_pool.cc:227] Handling call
0808 13:34:50.077375 (+1845037us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:34:50.089701 5200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:34:50.090560 5200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:50.096565 4856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:34:50.097637 4856 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e8b888041b4135af55792050defcb5 in term 0.
I20260808 13:34:50.099804 5058 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5; no voters:
I20260808 13:34:50.112109 5205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:34:50.112792 5205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:34:50.113140 5205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:34:50.121605 5200 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:34:50.123024 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:50.124204 4980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e8b888041b4135af55792050defcb5 in term 0.
I20260808 13:34:50.136188 5205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:50.151605 5205 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 1 election: Requested vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:34:50.153666 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7"
I20260808 13:34:50.154438 4980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:34:50.150871 4856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c"
I20260808 13:34:50.159144 4856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:34:50.161942 5170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:34:50.162575 5170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:34:50.172312 4980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87e8b888041b4135af55792050defcb5 in term 1.
I20260808 13:34:50.175562 5061 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5; no voters:
I20260808 13:34:50.176662 4856 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87e8b888041b4135af55792050defcb5 in term 1.
I20260808 13:34:50.177048 5205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:34:50.178555 5205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 1 LEADER]: Becoming Leader. State: Replica: 87e8b888041b4135af55792050defcb5, State: Running, Role: LEADER
I20260808 13:34:50.183138 5205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:50.203253 5200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87e8b888041b4135af55792050defcb5. Latest consensus state: current_term: 1 leader_uuid: "87e8b888041b4135af55792050defcb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:50.204490 5200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: LEADER
I20260808 13:34:50.217317 5221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260808 13:34:50.236068 5209 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:34:50.238351 5209 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:34:50.244657 5225 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 13:34:50.246493 5221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:34:50.206332 4791 master_runner.cc:186] Error getting master registration for 127.4.170.61:46555: OK, instance_id { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" instance_seqno: 1786196085190411 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:34:50.281369 4791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:34:50.296352 4980 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Refusing update from remote peer 87e8b888041b4135af55792050defcb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:34:50.299026 4856 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Refusing update from remote peer 87e8b888041b4135af55792050defcb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:34:50.301563 5200 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:34:50.306260 5205 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:34:50.315995 4947 tablet.cc:2369] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:34:50.360005 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87e8b888041b4135af55792050defcb5. Latest consensus state: current_term: 1 leader_uuid: "87e8b888041b4135af55792050defcb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:50.361423 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:50.371804 5198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87e8b888041b4135af55792050defcb5. Latest consensus state: current_term: 1 leader_uuid: "87e8b888041b4135af55792050defcb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:50.372964 5198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:50.404608 5229 mvcc.cc:205] Tried to move back new op lower bound from 7316259185825636352 to 7316259185407725568. Current Snapshot: MvccSnapshot[applied={T|T < 7316259185825636352}]
I20260808 13:34:50.406584 5205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87e8b888041b4135af55792050defcb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:50.408098 5205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: LEADER
I20260808 13:34:50.405275 5200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87e8b888041b4135af55792050defcb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:50.409988 5200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: LEADER
I20260808 13:34:50.413647 5221 catalog_manager.cc:1347] Generated new cluster ID: 9d0f658ea6c34f378fe8b1ad811bf2bf
I20260808 13:34:50.414074 5221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:34:50.416412 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87e8b888041b4135af55792050defcb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:50.417407 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:50.420651 5198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87e8b888041b4135af55792050defcb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:50.421770 5198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:50.464679 5221 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:34:50.469312 5221 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:34:50.518702 5221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Generated new TSK 0
I20260808 13:34:50.529556 5221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:34:50.588625 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.170.1:0
--local_ip_for_outbound_sockets=127.4.170.1
--webserver_interface=127.4.170.1
--webserver_port=0
--tserver_master_addrs=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--builtin_ntp_servers=127.4.170.20:42231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260808 13:34:51.218935 5241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:34:51.220139 5241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:34:51.220865 5241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 13:34:51.246685 5209 catalog_manager.cc:1259] Loaded cluster ID: 9d0f658ea6c34f378fe8b1ad811bf2bf
I20260808 13:34:51.247196 5209 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: loading cluster ID for follower catalog manager: success
I20260808 13:34:51.257609 5209 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: acquiring CA information for follower catalog manager: success
I20260808 13:34:51.277729 5225 catalog_manager.cc:1259] Loaded cluster ID: 9d0f658ea6c34f378fe8b1ad811bf2bf
I20260808 13:34:51.278154 5225 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: loading cluster ID for follower catalog manager: success
I20260808 13:34:51.300346 5225 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: acquiring CA information for follower catalog manager: success
I20260808 13:34:51.308463 5209 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 13:34:51.309552 5241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:34:51.311219 5241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.170.1
I20260808 13:34:51.318658 5225 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:34:51.417899 5241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.170.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.170.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.170.1
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:34:51.419795 5241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:34:51.422149 5241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:34:51.487080 5255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:34:51.934875 5260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:51.936935 5260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:51.941941 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:51.941864 4856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:34:51.944247 4941 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5
I20260808 13:34:51.946115 5260 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:51.947347 5260 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:34:52.333062 5264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:52.333940 5264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:52.338335 5264 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:52.340631 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:52.340631 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:52.342957 4820 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:34:52.344698 5264 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:34:52.571558 5265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:52.574736 5265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:52.580693 4856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:34:52.585718 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:52.591291 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5
I20260808 13:34:52.598330 5265 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:52.603700 5265 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:34:52.885864 5267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:52.886834 5267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:52.890939 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:52.893754 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:52.895803 5267 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:52.897817 4820 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:34:52.900022 5267 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260808 13:34:52.855013 5253 debug-util.cc:398] Leaking SignalData structure 0x7b08000148c0 after lost signal to thread 5241
W20260808 13:34:53.371507 5253 kernel_stack_watchdog.cc:198] Thread 5241 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 372ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260808 13:34:51.491880 5254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:34:53.374084 5241 thread.cc:636] GCE (cloud detector) Time spent creating pthread: real 1.893s user 0.642s sys 0.996s
W20260808 13:34:53.374490 5241 thread.cc:602] GCE (cloud detector) Time spent starting thread: real 1.894s user 0.642s sys 0.996s
I20260808 13:34:53.387527 5241 server_base.cc:1034] running on GCE node
W20260808 13:34:53.388751 5273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:34:53.390519 5241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:34:53.393680 5241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:34:53.395165 5241 hybrid_clock.cc:648] HybridClock initialized: now 1786196093395118 us; error 45 us; skew 500 ppm
I20260808 13:34:53.396462 5241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:34:53.416599 5241 webserver.cc:460] Webserver started at http://127.4.170.1:40121/ using document root <none> and password file <none>
I20260808 13:34:53.417991 5241 fs_manager.cc:359] Metadata directory not provided
I20260808 13:34:53.418295 5241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:34:53.418951 5241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:34:53.426481 5241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/data/instance:
uuid: "946c9d8462644deba5b45fa724f1923c"
format_stamp: "Formatted at 2026-08-08 13:34:53 on dist-test-slave-twwt"
I20260808 13:34:53.428257 5241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/instance:
uuid: "946c9d8462644deba5b45fa724f1923c"
format_stamp: "Formatted at 2026-08-08 13:34:53 on dist-test-slave-twwt"
I20260808 13:34:53.438812 5241 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.007s sys 0.004s
I20260808 13:34:53.445921 5281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:34:53.447234 5241 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.002s
I20260808 13:34:53.447628 5241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal
uuid: "946c9d8462644deba5b45fa724f1923c"
format_stamp: "Formatted at 2026-08-08 13:34:53 on dist-test-slave-twwt"
I20260808 13:34:53.448076 5241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:34:53.507722 5241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:34:53.509984 5241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:34:53.510769 5241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:34:53.519913 5241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:34:53.531167 5241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:34:53.531446 5241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:34:53.531750 5241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:34:53.531989 5241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:34:53.898988 5398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:53.899864 5398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:53.906106 5398 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:53.905282 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:53.909569 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:53.912060 4823 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:34:53.913647 5398 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:34:54.100921 5421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:54.103518 5421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:54.117519 4856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:34:54.130463 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:54.150970 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5
I20260808 13:34:54.157521 5421 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:54.159155 5421 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:34:54.389880 5460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.1:40797 every 8 connection(s)
I20260808 13:34:54.390651 5241 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.1:40797
I20260808 13:34:54.395949 5241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/data/info.pb
I20260808 13:34:54.417176 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 5241
I20260808 13:34:54.419283 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/instance
I20260808 13:34:54.447747 5470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:54.449514 5470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:54.519167 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:54.527976 5470 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:54.532306 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
W20260808 13:34:54.540447 4784 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.ConnectToMaster -> {remote=127.4.170.62:34069, user_credentials={real_user=slave}} blocked reactor thread for 60056.5us
I20260808 13:34:54.546399 4823 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:34:54.554958 5470 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:34:54.579886 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.170.2:0
--local_ip_for_outbound_sockets=127.4.170.2
--webserver_interface=127.4.170.2
--webserver_port=0
--tserver_master_addrs=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--builtin_ntp_servers=127.4.170.20:42231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260808 13:34:54.727598 5463 heartbeater.cc:347] Connected to a master server at 127.4.170.61:46555
I20260808 13:34:54.728721 5463 heartbeater.cc:464] Registering TS with master...
I20260808 13:34:54.732103 5463 heartbeater.cc:511] Master 127.4.170.61:46555 requested a full tablet report, sending...
I20260808 13:34:54.744328 4965 ts_manager.cc:194] Registered new tserver with Master: 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:34:54.745257 5476 heartbeater.cc:347] Connected to a master server at 127.4.170.62:34069
I20260808 13:34:54.746311 5476 heartbeater.cc:464] Registering TS with master...
I20260808 13:34:54.749644 5476 heartbeater.cc:511] Master 127.4.170.62:34069 requested a full tablet report, sending...
I20260808 13:34:54.756825 5462 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:34:54.757843 5462 heartbeater.cc:464] Registering TS with master...
I20260808 13:34:54.760561 4842 ts_manager.cc:194] Registered new tserver with Master: 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:34:54.761675 5462 heartbeater.cc:511] Master 127.4.170.60:37135 requested a full tablet report, sending...
I20260808 13:34:54.769183 5087 ts_manager.cc:194] Registered new tserver with Master: 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:34:54.778398 5087 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.170.1:58055
I20260808 13:34:55.153616 5486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:55.154556 5486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:55.159930 5486 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:55.161379 4856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:34:55.164047 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:55.167006 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5
I20260808 13:34:55.169494 5486 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:34:55.293375 5462 heartbeater.cc:503] Master 127.4.170.60:37135 was elected leader, sending a full tablet report...
I20260808 13:34:55.500263 5488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:55.501703 5488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:55.506556 5488 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:55.509737 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:55.509958 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:55.514145 4823 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:34:55.516172 5488 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260808 13:34:55.719600 5485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:34:55.720830 5485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:34:55.722716 5485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:34:55.800791 5485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:34:55.804615 5485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.170.2
I20260808 13:34:55.895227 5485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.170.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.170.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.170.2
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:34:55.898275 5485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:34:55.902014 5485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:34:55.955417 5501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:34:57.118415 5510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:57.246243 5511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:57.150740 5510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:57.247304 5511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:57.255592 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:34:57.260514 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:57.264940 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5
I20260808 13:34:57.267988 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:57.269513 4980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e672d7386144b299688abea9c64854c in term 1.
I20260808 13:34:57.270269 5511 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:57.273242 5511 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:34:57.273078 4820 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters:
I20260808 13:34:57.280393 5510 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:57.280467 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:57.281263 5510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:34:57.288056 5510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:57.288434 5510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:34:57.367236 5510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:57.399150 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7"
I20260808 13:34:57.406476 4864 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 2 FOLLOWER]: Rejecting Update request from peer 87e8b888041b4135af55792050defcb5 for earlier term 1. Current term is 2. Ops: []
I20260808 13:34:57.420869 5235 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260808 13:34:57.431818 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5"
I20260808 13:34:57.436299 4823 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:34:57.438356 5515 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 13:34:57.450378 5517 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 1 LEADER]: Stepping down as leader of term 1
I20260808 13:34:57.451269 5517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 87e8b888041b4135af55792050defcb5, State: Running, Role: LEADER
I20260808 13:34:57.452894 5517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:57.455619 5517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:34:57.449654 5510 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 2 election: Requested vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:57.839089 5520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:57.844262 5520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:57.865008 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:34:57.867431 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:57.868043 4864 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 63c875fc5bcd41a9b1bcf3bd4ae321d7 in current term 2: Already voted for candidate 3e672d7386144b299688abea9c64854c in this term.
I20260808 13:34:57.869171 5103 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63c875fc5bcd41a9b1bcf3bd4ae321d7 in term 2.
I20260808 13:34:57.873577 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c
I20260808 13:34:57.878568 5520 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:57.881656 5520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:34:57.887323 5520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 87e8b888041b4135af55792050defcb5)
I20260808 13:34:57.890054 5520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:34:57.965286 5520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:57.999327 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c"
I20260808 13:34:58.005234 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5"
I20260808 13:34:58.002353 4864 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 63c875fc5bcd41a9b1bcf3bd4ae321d7 in current term 2: Already voted for candidate 3e672d7386144b299688abea9c64854c in this term.
I20260808 13:34:58.019057 5520 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 election: Requested vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:58.104791 5103 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63c875fc5bcd41a9b1bcf3bd4ae321d7 in term 2.
I20260808 13:34:58.118206 5522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:34:58.119462 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c
I20260808 13:34:58.120460 5522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:58.123226 5520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:34:58.125193 5520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 2 LEADER]: Becoming Leader. State: Replica: 63c875fc5bcd41a9b1bcf3bd4ae321d7, State: Running, Role: LEADER
I20260808 13:34:58.130782 5522 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:58.134151 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:58.138746 5520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:58.141907 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:58.146566 5103 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e672d7386144b299688abea9c64854c in term 2.
I20260808 13:34:58.151194 4823 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:34:58.153841 5522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 13:34:58.154579 5522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:34:58.155709 5522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:34:58.176672 5521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c875fc5bcd41a9b1bcf3bd4ae321d7. Latest consensus state: current_term: 2 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:58.191323 5521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:34:58.196110 5526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:34:58.217480 5522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:58.218983 5170 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:34:58.228382 5522 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 3 election: Requested vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:58.235514 5103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5"
I20260808 13:34:58.236793 5103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:34:58.251303 5526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:34:58.251902 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7"
I20260808 13:34:58.269914 5526 catalog_manager.cc:1259] Loaded cluster ID: 9d0f658ea6c34f378fe8b1ad811bf2bf
I20260808 13:34:58.272745 5526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:34:58.283972 5526 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:34:58.290970 5526 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: Loaded TSK: 0
I20260808 13:34:58.295550 5526 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:34:58.346385 5103 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e672d7386144b299688abea9c64854c in term 3.
I20260808 13:34:58.350135 4823 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:34:58.352352 5522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 3 FOLLOWER]: Leader election won for term 3
I20260808 13:34:58.366029 5522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 3 LEADER]: Becoming Leader. State: Replica: 3e672d7386144b299688abea9c64854c, State: Running, Role: LEADER
I20260808 13:34:58.368631 5522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:58.395687 5527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e672d7386144b299688abea9c64854c. Latest consensus state: current_term: 3 leader_uuid: "3e672d7386144b299688abea9c64854c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:58.400008 5527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: LEADER
I20260808 13:34:58.420439 5530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:34:58.457857 5530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:34:58.479372 5530 catalog_manager.cc:1259] Loaded cluster ID: 9d0f658ea6c34f378fe8b1ad811bf2bf
I20260808 13:34:58.479944 5530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:34:58.489434 5530 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:34:58.497433 5530 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Loaded TSK: 0
I20260808 13:34:58.501139 5530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:34:58.551426 5103 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Rejecting Update request from peer 63c875fc5bcd41a9b1bcf3bd4ae321d7 for earlier term 2. Current term is 3. Ops: []
I20260808 13:34:58.559365 4864 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 3 LEADER]: Rejecting Update request from peer 63c875fc5bcd41a9b1bcf3bd4ae321d7 for earlier term 2. Current term is 3. Ops: []
I20260808 13:34:58.565915 5520 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260808 13:34:58.613988 5520 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Peer 3e672d7386144b299688abea9c64854c log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260808 13:34:58.621750 5521 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 2 LEADER]: Stepping down as leader of term 2
I20260808 13:34:58.622906 5521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 63c875fc5bcd41a9b1bcf3bd4ae321d7, State: Running, Role: LEADER
I20260808 13:34:58.625275 5532 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260808 13:34:58.635483 5521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:58.642033 5521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:34:58.769380 4980 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Refusing update from remote peer 3e672d7386144b299688abea9c64854c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260808 13:34:58.771440 4980 pending_rounds.cc:76] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: Aborting all ops after (but not including) 4
I20260808 13:34:58.772881 4980 pending_rounds.cc:97] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260808 13:34:58.774497 4980 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260808 13:34:58.780818 5527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [LEADER]: Connected to new peer: Peer: permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260808 13:34:58.796906 5103 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Refusing update from remote peer 3e672d7386144b299688abea9c64854c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260808 13:34:58.802358 5533 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260808 13:34:58.806555 5521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e672d7386144b299688abea9c64854c. Latest consensus state: current_term: 3 leader_uuid: "3e672d7386144b299688abea9c64854c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:58.810163 5521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:58.824074 5463 heartbeater.cc:503] Master 127.4.170.61:46555 was elected leader, sending a full tablet report...
I20260808 13:34:58.838845 5527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "3e672d7386144b299688abea9c64854c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:58.840351 5527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: LEADER
I20260808 13:34:58.841015 5521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "3e672d7386144b299688abea9c64854c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:58.842880 5521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:58.842447 5534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e672d7386144b299688abea9c64854c. Latest consensus state: current_term: 3 leader_uuid: "3e672d7386144b299688abea9c64854c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:58.843972 5534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:58.848857 5527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "3e672d7386144b299688abea9c64854c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:58.850582 5527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: LEADER
I20260808 13:34:58.862354 5534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "3e672d7386144b299688abea9c64854c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:34:58.864444 5534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:34:59.317485 5521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:34:59.318286 5521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:59.325191 5102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:59.325492 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:34:59.331202 5521 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:59.331915 4941 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5
I20260808 13:34:59.336719 5521 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:34:59.429605 5476 heartbeater.cc:503] Master 127.4.170.62:34069 was elected leader, sending a full tablet report...
W20260808 13:34:57.353756 5498 debug-util.cc:398] Leaking SignalData structure 0x7b08000148c0 after lost signal to thread 5485
W20260808 13:34:55.956905 5502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:34:59.572018 5498 kernel_stack_watchdog.cc:198] Thread 5485 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 395ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260808 13:34:59.573258 5485 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 3.625s user 0.989s sys 1.748s
W20260808 13:34:59.573545 5485 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 3.625s user 0.989s sys 1.748s
W20260808 13:34:59.575438 5506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:34:59.581254 5503 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 3629 milliseconds
I20260808 13:34:59.581254 5485 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260808 13:34:59.582994 5485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:34:59.585233 5485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:34:59.586663 5485 hybrid_clock.cc:648] HybridClock initialized: now 1786196099586602 us; error 50 us; skew 500 ppm
I20260808 13:34:59.587638 5485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:34:59.598984 5485 webserver.cc:460] Webserver started at http://127.4.170.2:40687/ using document root <none> and password file <none>
I20260808 13:34:59.600057 5485 fs_manager.cc:359] Metadata directory not provided
I20260808 13:34:59.600306 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:34:59.600781 5485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:34:59.606228 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/data/instance:
uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
format_stamp: "Formatted at 2026-08-08 13:34:59 on dist-test-slave-twwt"
I20260808 13:34:59.607471 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/instance:
uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
format_stamp: "Formatted at 2026-08-08 13:34:59 on dist-test-slave-twwt"
I20260808 13:34:59.615697 5485 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.003s
I20260808 13:34:59.621635 5549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:34:59.622879 5485 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 13:34:59.623248 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal
uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
format_stamp: "Formatted at 2026-08-08 13:34:59 on dist-test-slave-twwt"
I20260808 13:34:59.623647 5485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:34:59.667869 5485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:34:59.669502 5485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:34:59.669999 5485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:34:59.672580 5485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:34:59.678589 5485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:34:59.678881 5485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 13:34:59.679141 5485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:34:59.679322 5485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:34:59.783193 5584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:34:59.784442 5584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:59.789481 5584 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:34:59.791105 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:34:59.792403 4978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:34:59.795722 5061 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:34:59.797343 5584 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:34:59.877032 5604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:34:59.878265 5604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:34:59.886822 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:34:59.893213 5102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:34:59.895833 5604 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:34:59.896106 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5
I20260808 13:34:59.898561 5604 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:00.306923 5485 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.2:40577
I20260808 13:35:00.307104 5719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.2:40577 every 8 connection(s)
I20260808 13:35:00.309891 5485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/data/info.pb
I20260808 13:35:00.318681 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 5485
I20260808 13:35:00.319466 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/instance
I20260808 13:35:00.338195 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.170.3:0
--local_ip_for_outbound_sockets=127.4.170.3
--webserver_interface=127.4.170.3
--webserver_port=0
--tserver_master_addrs=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--builtin_ntp_servers=127.4.170.20:42231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260808 13:35:00.396044 5728 heartbeater.cc:347] Connected to a master server at 127.4.170.62:34069
I20260808 13:35:00.396581 5728 heartbeater.cc:464] Registering TS with master...
I20260808 13:35:00.398334 5728 heartbeater.cc:511] Master 127.4.170.62:34069 requested a full tablet report, sending...
I20260808 13:35:00.398892 5721 heartbeater.cc:347] Connected to a master server at 127.4.170.61:46555
I20260808 13:35:00.399299 5721 heartbeater.cc:464] Registering TS with master...
I20260808 13:35:00.400465 5721 heartbeater.cc:511] Master 127.4.170.61:46555 requested a full tablet report, sending...
I20260808 13:35:00.401502 5720 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:35:00.401648 4842 ts_manager.cc:194] Registered new tserver with Master: 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:00.401890 5720 heartbeater.cc:464] Registering TS with master...
I20260808 13:35:00.402977 5720 heartbeater.cc:511] Master 127.4.170.60:37135 requested a full tablet report, sending...
I20260808 13:35:00.404393 4842 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.170.2:55071
I20260808 13:35:00.406100 4965 ts_manager.cc:194] Registered new tserver with Master: 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:00.407037 5087 ts_manager.cc:194] Registered new tserver with Master: 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:00.422436 5738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:00.423108 5738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:00.426030 5738 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:35:00.427225 5102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:35:00.427166 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:00.429141 4941 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5
I20260808 13:35:00.430076 5738 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260808 13:35:00.864563 5737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:35:00.865213 5737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:35:00.865840 5737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 13:35:00.911607 5728 heartbeater.cc:503] Master 127.4.170.62:34069 was elected leader, sending a full tablet report...
W20260808 13:35:00.913756 5737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:35:00.914855 5737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.170.3
I20260808 13:35:00.953572 5737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.170.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.170.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.170.3
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:35:00.954849 5737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:35:00.956506 5737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:35:00.988830 5747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:35:01.727962 5759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:01.729036 5759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:01.736131 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:01.746276 5759 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:01.746567 4978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:01.750900 5061 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:01.753422 5759 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:01.835528 5760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:01.836330 5760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:01.840111 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:01.853639 5760 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:35:01.855393 5102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:35:01.857442 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5
I20260808 13:35:01.859102 5760 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:02.338768 5761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:02.341650 5761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:02.351459 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:02.361841 4978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:02.364161 5061 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:02.374825 5761 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:02.377399 5761 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260808 13:35:02.380911 5744 debug-util.cc:398] Leaking SignalData structure 0x7b08000148c0 after lost signal to thread 5737
W20260808 13:35:02.634404 5744 kernel_stack_watchdog.cc:198] Thread 5737 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 391ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260808 13:35:00.988950 5748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:35:02.636058 5737 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 1.650s user 0.002s sys 0.003s
W20260808 13:35:02.636452 5737 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 1.650s user 0.002s sys 0.003s
W20260808 13:35:02.638788 5750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:35:02.640383 5749 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1644 milliseconds
I20260808 13:35:02.640448 5737 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260808 13:35:02.641932 5737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:35:02.645195 5737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:35:02.646574 5737 hybrid_clock.cc:648] HybridClock initialized: now 1786196102646530 us; error 43 us; skew 500 ppm
I20260808 13:35:02.647609 5737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:35:02.657397 5737 webserver.cc:460] Webserver started at http://127.4.170.3:39551/ using document root <none> and password file <none>
I20260808 13:35:02.658535 5737 fs_manager.cc:359] Metadata directory not provided
I20260808 13:35:02.658876 5737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:35:02.659459 5737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:35:02.665381 5737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/data/instance:
uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
format_stamp: "Formatted at 2026-08-08 13:35:02 on dist-test-slave-twwt"
I20260808 13:35:02.666982 5737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/instance:
uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
format_stamp: "Formatted at 2026-08-08 13:35:02 on dist-test-slave-twwt"
I20260808 13:35:02.677083 5737 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.004s
I20260808 13:35:02.685672 5768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:35:02.687090 5737 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260808 13:35:02.687462 5737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal
uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
format_stamp: "Formatted at 2026-08-08 13:35:02 on dist-test-slave-twwt"
I20260808 13:35:02.687930 5737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:35:02.750802 5737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:35:02.752813 5737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:35:02.753398 5737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:35:02.757741 5737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:35:02.764643 5737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:35:02.764945 5737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:35:02.765219 5737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:35:02.765403 5737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:35:02.813498 5814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:02.814091 5814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:02.816998 5814 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:35:02.817044 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:02.817777 5102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:35:02.819111 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5
I20260808 13:35:02.819841 5814 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:02.889714 5865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:02.890395 5865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:02.894083 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:02.894301 4978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:02.896126 5865 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:02.896266 5058 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:02.897655 5865 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:03.130340 5737 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.3:34173
I20260808 13:35:03.130538 5947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.3:34173 every 8 connection(s)
I20260808 13:35:03.134088 5737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/data/info.pb
I20260808 13:35:03.140377 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 5737
I20260808 13:35:03.141753 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/instance
I20260808 13:35:03.245574 5948 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:35:03.246229 5948 heartbeater.cc:464] Registering TS with master...
I20260808 13:35:03.248155 5948 heartbeater.cc:511] Master 127.4.170.60:37135 requested a full tablet report, sending...
I20260808 13:35:03.251787 5950 heartbeater.cc:347] Connected to a master server at 127.4.170.61:46555
I20260808 13:35:03.252260 5950 heartbeater.cc:464] Registering TS with master...
I20260808 13:35:03.253438 5086 ts_manager.cc:194] Registered new tserver with Master: 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:03.253722 5950 heartbeater.cc:511] Master 127.4.170.61:46555 requested a full tablet report, sending...
I20260808 13:35:03.255772 5954 heartbeater.cc:347] Connected to a master server at 127.4.170.62:34069
I20260808 13:35:03.256167 5954 heartbeater.cc:464] Registering TS with master...
I20260808 13:35:03.257730 5954 heartbeater.cc:511] Master 127.4.170.62:34069 requested a full tablet report, sending...
I20260808 13:35:03.260861 4965 ts_manager.cc:194] Registered new tserver with Master: 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:03.264110 4842 ts_manager.cc:194] Registered new tserver with Master: 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:03.266744 4842 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.170.3:51951
I20260808 13:35:03.294183 4776 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:35:03.415333 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_e000c4b86cd843c2acc30310c1127f77"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:03.418411 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_9cf691b142a44418a8ace03e1a372f96"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:03.421921 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e000c4b86cd843c2acc30310c1127f77 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:03.421952 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9cf691b142a44418a8ace03e1a372f96 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:03.432914 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_e3f5759eb9a0408894ab39df2973fb69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:03.437295 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3f5759eb9a0408894ab39df2973fb69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:03.445962 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5e4a0bfe70944fb9ba35294f68db0e53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:03.448915 5979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:03.449709 5979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:03.453531 5979 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:03.456503 4978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:03.460233 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:03.463630 5058 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
W20260808 13:35:03.467031 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e4a0bfe70944fb9ba35294f68db0e53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:03.471056 5979 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:03.564354 4776 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu with pid 5020
I20260808 13:35:03.572023 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_e000c4b86cd843c2acc30310c1127f77" } new_table_name: "default.Table_a297438a46d34ceaac3836c97edd6f38" modify_external_catalogs: true
W20260808 13:35:03.617560 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f0a957251a6540c9b4a83a3517eb4ed6 has no consensus state
I20260808 13:35:03.620616 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_e3f5759eb9a0408894ab39df2973fb69" } new_table_name: "default.Table_a93c50ca76134d7caf37830d9f266335" modify_external_catalogs: true
I20260808 13:35:03.622212 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_9cf691b142a44418a8ace03e1a372f96" } modify_external_catalogs: true
I20260808 13:35:03.622913 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_9cf691b142a44418a8ace03e1a372f96" } modify_external_catalogs: true
W20260808 13:35:03.623402 4823 connection.cc:535] client connection to 127.4.170.60:37135 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:35:03.623831 4823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 14 similar messages]
I20260808 13:35:03.633114 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5e4a0bfe70944fb9ba35294f68db0e53" } modify_external_catalogs: true
I20260808 13:35:03.633718 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5e4a0bfe70944fb9ba35294f68db0e53" } modify_external_catalogs: true
W20260808 13:35:03.670723 4821 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet f0a957251a6540c9b4a83a3517eb4ed6 (table default.Table_a297438a46d34ceaac3836c97edd6f38 [id=462ae7e5fbb8461ebd32fa4cd72c941e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f0a957251a6540c9b4a83a3517eb4ed6 consensus state has no leader
I20260808 13:35:03.694253 4842 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 5c2ed27575ce4fcca1765f115f4dda84
I20260808 13:35:03.695497 5616 tablet_service.cc:1467] Processing CreateTablet for tablet f0a957251a6540c9b4a83a3517eb4ed6 (DEFAULT_TABLE table=default.Table_a297438a46d34ceaac3836c97edd6f38 [id=462ae7e5fbb8461ebd32fa4cd72c941e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:03.699416 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0a957251a6540c9b4a83a3517eb4ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:03.705061 5845 tablet_service.cc:1467] Processing CreateTablet for tablet f0a957251a6540c9b4a83a3517eb4ed6 (DEFAULT_TABLE table=default.Table_a297438a46d34ceaac3836c97edd6f38 [id=462ae7e5fbb8461ebd32fa4cd72c941e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:03.707477 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0a957251a6540c9b4a83a3517eb4ed6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:03.718086 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5f18e5b0a4c445a88b7717441edb323f has no consensus state
I20260808 13:35:03.721127 4840 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1cf42fc5bda74958abb8823eb6ff2197
W20260808 13:35:03.721964 4823 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c -> Peer 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135): Couldn't send request to peer 87e8b888041b4135af55792050defcb5. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:03.741019 5462 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:37135 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:35:03.756016 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_9f2546f5de354f2d9b57f67621ef4305"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:03.757818 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f2546f5de354f2d9b57f67621ef4305 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:03.761379 5615 tablet_service.cc:1467] Processing CreateTablet for tablet 5f18e5b0a4c445a88b7717441edb323f (DEFAULT_TABLE table=default.Table_a93c50ca76134d7caf37830d9f266335 [id=3540defcda954120ba76fe9e4c3869a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:03.762650 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f18e5b0a4c445a88b7717441edb323f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:03.769068 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 5f18e5b0a4c445a88b7717441edb323f (DEFAULT_TABLE table=default.Table_a93c50ca76134d7caf37830d9f266335 [id=3540defcda954120ba76fe9e4c3869a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:03.770646 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f18e5b0a4c445a88b7717441edb323f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:03.773825 5954 heartbeater.cc:503] Master 127.4.170.62:34069 was elected leader, sending a full tablet report...
W20260808 13:35:03.792771 5948 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:37135 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
I20260808 13:35:03.793115 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 5f18e5b0a4c445a88b7717441edb323f (DEFAULT_TABLE table=default.Table_a93c50ca76134d7caf37830d9f266335 [id=3540defcda954120ba76fe9e4c3869a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:03.806248 6026 tablet_bootstrap.cc:492] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:03.813408 6033 tablet_bootstrap.cc:492] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:03.815629 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c08ec3a916d44f25b61fa57b88efcbcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:03.819137 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_1ab91b8bd91d4fb9af1ffc1e9c6c7c5a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:03.821599 6026 tablet_bootstrap.cc:654] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:03.825663 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c08ec3a916d44f25b61fa57b88efcbcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:03.827711 6026 log.cc:824] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Log is configured to *not* fsync() on all Append() calls
I20260808 13:35:03.835459 6033 tablet_bootstrap.cc:654] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:03.798909 5357 tablet_service.cc:1467] Processing CreateTablet for tablet f0a957251a6540c9b4a83a3517eb4ed6 (DEFAULT_TABLE table=default.Table_a297438a46d34ceaac3836c97edd6f38 [id=462ae7e5fbb8461ebd32fa4cd72c941e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:03.839294 6033 log.cc:824] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Log is configured to *not* fsync() on all Append() calls
I20260808 13:35:03.840974 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0a957251a6540c9b4a83a3517eb4ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:03.845389 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f18e5b0a4c445a88b7717441edb323f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:03.843199 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5f0e676e2cf14d31b0439a4fdd39a58f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:03.844848 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a93c50ca76134d7caf37830d9f266335" } new_table_name: "default.Table_b8444ca9c0824bea894226173b44e0f0" modify_external_catalogs: true
W20260808 13:35:03.853897 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ab91b8bd91d4fb9af1ffc1e9c6c7c5a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:03.860620 6033 tablet_bootstrap.cc:492] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:03.861497 6033 ts_tablet_manager.cc:1397] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.049s user 0.025s sys 0.017s
W20260808 13:35:03.858304 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f0e676e2cf14d31b0439a4fdd39a58f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:03.908298 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a297438a46d34ceaac3836c97edd6f38" } modify_external_catalogs: true
I20260808 13:35:03.909448 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a297438a46d34ceaac3836c97edd6f38" } modify_external_catalogs: true
I20260808 13:35:03.920433 6033 raft_consensus.cc:348] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:03.922075 6033 raft_consensus.cc:374] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:03.925957 6033 raft_consensus.cc:729] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:03.926191 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_9f2546f5de354f2d9b57f67621ef4305" } new_table_name: "default.Table_1ccce545a53c4f4ca79ede2bee9ebceb" modify_external_catalogs: true
I20260808 13:35:03.928397 6033 consensus_queue.cc:260] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:03.944859 6033 ts_tablet_manager.cc:1428] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.083s user 0.050s sys 0.013s
W20260808 13:35:03.956660 5958 tablet.cc:2369] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:35:03.966249 6026 tablet_bootstrap.cc:492] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:03.967103 6026 ts_tablet_manager.cc:1397] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.162s user 0.056s sys 0.086s
I20260808 13:35:03.985778 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 38aab2b333884120b1851c8cdf93d140 (DEFAULT_TABLE table=default.Table_9f2546f5de354f2d9b57f67621ef4305 [id=7d8eeac8f7374777a4be8b441045af2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:03.986599 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 38aab2b333884120b1851c8cdf93d140 (DEFAULT_TABLE table=default.Table_9f2546f5de354f2d9b57f67621ef4305 [id=7d8eeac8f7374777a4be8b441045af2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:03.989373 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38aab2b333884120b1851c8cdf93d140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:03.999709 4839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1ab91b8bd91d4fb9af1ffc1e9c6c7c5a" } modify_external_catalogs: true
I20260808 13:35:04.000806 4839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1ab91b8bd91d4fb9af1ffc1e9c6c7c5a" } modify_external_catalogs: true
I20260808 13:35:04.003427 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 38aab2b333884120b1851c8cdf93d140 (DEFAULT_TABLE table=default.Table_9f2546f5de354f2d9b57f67621ef4305 [id=7d8eeac8f7374777a4be8b441045af2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.005108 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38aab2b333884120b1851c8cdf93d140. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:04.023576 5720 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:37135 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111)
W20260808 13:35:04.025457 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5f18e5b0a4c445a88b7717441edb323f consensus state has no leader
I20260808 13:35:04.027273 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38aab2b333884120b1851c8cdf93d140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.070284 6033 tablet_bootstrap.cc:492] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:04.077442 5533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "3e672d7386144b299688abea9c64854c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:35:04.079610 5533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: LEADER
I20260808 13:35:04.083425 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_89ea302fae144a61a691c2d916820cd8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:04.086102 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89ea302fae144a61a691c2d916820cd8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:04.098013 6046 tablet_bootstrap.cc:492] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:04.091580 6033 tablet_bootstrap.cc:654] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.105000 6026 raft_consensus.cc:348] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:04.106467 6026 raft_consensus.cc:374] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.110877 6026 raft_consensus.cc:729] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.112278 6026 consensus_queue.cc:260] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:04.143149 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c64b1240bf04403d83f297ac2d40590c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:04.147508 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c64b1240bf04403d83f297ac2d40590c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:04.147804 6033 tablet_bootstrap.cc:492] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:04.148407 6033 ts_tablet_manager.cc:1397] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.078s user 0.005s sys 0.019s
I20260808 13:35:04.163229 6026 ts_tablet_manager.cc:1428] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.196s user 0.055s sys 0.021s
I20260808 13:35:04.163266 6033 raft_consensus.cc:348] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.164654 6033 raft_consensus.cc:374] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.165145 6033 raft_consensus.cc:729] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.165714 6026 tablet_bootstrap.cc:492] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:04.166496 6033 consensus_queue.cc:260] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.171444 6033 ts_tablet_manager.cc:1428] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.023s user 0.006s sys 0.009s
W20260808 13:35:04.176541 4837 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38aab2b333884120b1851c8cdf93d140 consensus state has no leader
I20260808 13:35:04.176729 6026 tablet_bootstrap.cc:654] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.181180 6033 tablet_bootstrap.cc:492] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:04.188265 6052 raft_consensus.cc:482] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:04.189126 6046 tablet_bootstrap.cc:654] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.189568 6052 raft_consensus.cc:504] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:04.193845 6046 log.cc:824] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: Log is configured to *not* fsync() on all Append() calls
I20260808 13:35:04.202519 6046 tablet_bootstrap.cc:492] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:04.203310 6026 tablet_bootstrap.cc:492] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:04.203542 6046 ts_tablet_manager.cc:1397] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.106s user 0.006s sys 0.024s
I20260808 13:35:04.203747 6026 ts_tablet_manager.cc:1397] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.038s user 0.027s sys 0.002s
I20260808 13:35:04.207185 6033 tablet_bootstrap.cc:654] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.206715 6026 raft_consensus.cc:348] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.207466 6026 raft_consensus.cc:374] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.207753 6026 raft_consensus.cc:729] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.218544 6026 consensus_queue.cc:260] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.220856 6026 ts_tablet_manager.cc:1428] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260808 13:35:04.221725 6026 tablet_bootstrap.cc:492] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:04.227831 6052 leader_election.cc:288] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:04.233892 4838 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b8444ca9c0824bea894226173b44e0f0" } new_table_name: "default.Table_e79e02d050604203b805744df0be2c8c" modify_external_catalogs: true
I20260808 13:35:04.227867 6026 tablet_bootstrap.cc:654] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.250077 6046 raft_consensus.cc:348] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.251334 6046 raft_consensus.cc:374] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.251675 6046 raft_consensus.cc:729] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.252755 6046 consensus_queue.cc:260] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.263346 6026 tablet_bootstrap.cc:492] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:04.263808 6026 ts_tablet_manager.cc:1397] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.042s user 0.011s sys 0.000s
I20260808 13:35:04.263818 6046 ts_tablet_manager.cc:1428] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.060s user 0.036s sys 0.011s
I20260808 13:35:04.263955 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet f0a957251a6540c9b4a83a3517eb4ed6
I20260808 13:35:04.268249 5616 tablet_service.cc:1514] Processing DeleteTablet for tablet f0a957251a6540c9b4a83a3517eb4ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:03 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:04.271862 6026 raft_consensus.cc:348] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.272706 6026 raft_consensus.cc:374] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.273056 6026 raft_consensus.cc:729] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.273830 6026 consensus_queue.cc:260] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.281810 6046 tablet_bootstrap.cc:492] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:04.283071 6026 ts_tablet_manager.cc:1428] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.019s user 0.008s sys 0.000s
I20260808 13:35:04.283864 5615 tablet_service.cc:1467] Processing CreateTablet for tablet ae5487d6a4204a6f8b97a1d09d1314ab (DEFAULT_TABLE table=default.Table_1ab91b8bd91d4fb9af1ffc1e9c6c7c5a [id=490c4641a4554492a415f6b987c72300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.284619 5845 tablet_service.cc:1514] Processing DeleteTablet for tablet f0a957251a6540c9b4a83a3517eb4ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:03 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:04.285048 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae5487d6a4204a6f8b97a1d09d1314ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.289685 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet f0a957251a6540c9b4a83a3517eb4ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:03 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:04.291090 5844 tablet_service.cc:1467] Processing CreateTablet for tablet ae5487d6a4204a6f8b97a1d09d1314ab (DEFAULT_TABLE table=default.Table_1ab91b8bd91d4fb9af1ffc1e9c6c7c5a [id=490c4641a4554492a415f6b987c72300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.292335 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae5487d6a4204a6f8b97a1d09d1314ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.292483 5357 tablet_service.cc:1467] Processing CreateTablet for tablet ae5487d6a4204a6f8b97a1d09d1314ab (DEFAULT_TABLE table=default.Table_1ab91b8bd91d4fb9af1ffc1e9c6c7c5a [id=490c4641a4554492a415f6b987c72300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.293550 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae5487d6a4204a6f8b97a1d09d1314ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.307500 6072 tablet_replica.cc:330] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:04.308372 6072 raft_consensus.cc:2229] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:04.308912 6072 raft_consensus.cc:2258] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:04.311715 6074 tablet_replica.cc:330] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:04.312927 6074 raft_consensus.cc:2229] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:04.313508 6074 raft_consensus.cc:2258] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:04.315241 4838 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5f18e5b0a4c445a88b7717441edb323f consensus state has no leader
I20260808 13:35:04.316181 6046 tablet_bootstrap.cc:654] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.320849 4840 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5f18e5b0a4c445a88b7717441edb323f (table default.Table_e79e02d050604203b805744df0be2c8c [id=3540defcda954120ba76fe9e4c3869a8]). Expected version 3 got 0
I20260808 13:35:04.324942 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 5f18e5b0a4c445a88b7717441edb323f (table default.Table_e79e02d050604203b805744df0be2c8c [id=3540defcda954120ba76fe9e4c3869a8]). Expected version 3 got 0
I20260808 13:35:04.326076 6072 ts_tablet_manager.cc:1905] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:04.333673 6074 ts_tablet_manager.cc:1905] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:04.337926 4842 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 5f18e5b0a4c445a88b7717441edb323f (table default.Table_e79e02d050604203b805744df0be2c8c [id=3540defcda954120ba76fe9e4c3869a8]). Expected version 3 got 0
I20260808 13:35:04.347496 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0a957251a6540c9b4a83a3517eb4ed6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:04.349889 6046 tablet_bootstrap.cc:492] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:04.350328 6046 ts_tablet_manager.cc:1397] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.069s user 0.011s sys 0.000s
W20260808 13:35:04.331240 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet f0a957251a6540c9b4a83a3517eb4ed6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f0a957251a6540c9b4a83a3517eb4ed6 already in progress: creating tablet
I20260808 13:35:04.353152 6046 raft_consensus.cc:348] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:04.353964 6046 raft_consensus.cc:374] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.354239 6046 raft_consensus.cc:729] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:35:04.356904 5550 leader_election.cc:341] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: Tablet not RUNNING: STOPPED
I20260808 13:35:04.361294 6046 consensus_queue.cc:260] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:04.363796 6046 ts_tablet_manager.cc:1428] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.013s user 0.008s sys 0.000s
I20260808 13:35:04.364704 6046 tablet_bootstrap.cc:492] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:04.373013 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet ae5487d6a4204a6f8b97a1d09d1314ab
I20260808 13:35:04.376776 6046 tablet_bootstrap.cc:654] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.379051 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet ae5487d6a4204a6f8b97a1d09d1314ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:04.381158 4839 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_89ea302fae144a61a691c2d916820cd8" } new_table_name: "default.Table_d83818a7ac294a0cbe2f914264d02c59" modify_external_catalogs: true
W20260808 13:35:04.383299 5484 tablet.cc:2369] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:35:04.386013 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet ae5487d6a4204a6f8b97a1d09d1314ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:04.385878 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_e4d8738a572f475a9becc0e4e90d6c1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:04.389122 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_7cd941437b494bdba3bc98e7bfad0c78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:04.391098 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4d8738a572f475a9becc0e4e90d6c1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:04.391314 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7cd941437b494bdba3bc98e7bfad0c78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:04.395910 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet ae5487d6a4204a6f8b97a1d09d1314ab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae5487d6a4204a6f8b97a1d09d1314ab already in progress: creating tablet
I20260808 13:35:04.408296 4840 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 38aab2b333884120b1851c8cdf93d140 (table default.Table_1ccce545a53c4f4ca79ede2bee9ebceb [id=7d8eeac8f7374777a4be8b441045af2d]). Expected version 1 got 0
I20260808 13:35:04.419973 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet f0a957251a6540c9b4a83a3517eb4ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:03 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:04.421039 6075 tablet_replica.cc:330] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:04.422226 6075 raft_consensus.cc:2229] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:04.422974 6075 raft_consensus.cc:2258] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:04.426258 6075 ts_tablet_manager.cc:1905] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:04.430915 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 78589.6us
W20260808 13:35:04.436791 5731 tablet.cc:2369] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:35:04.440251 5616 tablet_service.cc:1514] Processing DeleteTablet for tablet ae5487d6a4204a6f8b97a1d09d1314ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:04.441373 6052 raft_consensus.cc:482] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:04.441807 6052 raft_consensus.cc:504] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.443490 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0a957251a6540c9b4a83a3517eb4ed6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:04.446080 6052 leader_election.cc:288] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:04.447358 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38aab2b333884120b1851c8cdf93d140" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:04.447487 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38aab2b333884120b1851c8cdf93d140" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:04.448994 5550 leader_election.cc:341] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:04.451206 5551 leader_election.cc:341] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: Tablet not RUNNING: STOPPED
I20260808 13:35:04.450994 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_feaae5f9ed9e4419851beead826e589f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:04.451591 5551 leader_election.cc:302] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:04.452337 5551 raft_consensus.cc:2670] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260808 13:35:04.453158 5551 leader_election.cc:341] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:04.453439 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_feaae5f9ed9e4419851beead826e589f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:04.453527 5551 leader_election.cc:302] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:04.454051 6033 tablet_bootstrap.cc:492] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:04.454205 6052 raft_consensus.cc:2726] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:04.454537 6033 ts_tablet_manager.cc:1397] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.274s user 0.019s sys 0.010s
W20260808 13:35:04.474810 4839 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1a5a9f860b34797825be7568a0b4ccb has no consensus state
I20260808 13:35:04.490098 6046 tablet_bootstrap.cc:492] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:04.490571 6046 ts_tablet_manager.cc:1397] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.126s user 0.032s sys 0.053s
I20260808 13:35:04.491812 6033 raft_consensus.cc:348] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.492658 6033 raft_consensus.cc:374] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.493006 6033 raft_consensus.cc:729] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.493273 6046 raft_consensus.cc:348] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.493950 6046 raft_consensus.cc:374] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.494228 6046 raft_consensus.cc:729] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.493842 6033 consensus_queue.cc:260] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.494880 6046 consensus_queue.cc:260] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.499481 4839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c08ec3a916d44f25b61fa57b88efcbcc" } modify_external_catalogs: true
I20260808 13:35:04.500123 4839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c08ec3a916d44f25b61fa57b88efcbcc" } modify_external_catalogs: true
I20260808 13:35:04.500718 6046 ts_tablet_manager.cc:1428] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260808 13:35:04.501590 6046 tablet_bootstrap.cc:492] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:04.505183 6026 tablet_bootstrap.cc:492] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:04.510810 6033 ts_tablet_manager.cc:1428] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.056s user 0.013s sys 0.003s
I20260808 13:35:04.511183 6026 tablet_bootstrap.cc:654] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.515035 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_06f4a6ab47814783a9277ded39511f8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:04.516927 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06f4a6ab47814783a9277ded39511f8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:04.520757 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d83818a7ac294a0cbe2f914264d02c59" } new_table_name: "default.Table_18666f1bccbb4a41ba39aafa9354326b" modify_external_catalogs: true
I20260808 13:35:04.523391 6072 ts_tablet_manager.cc:1918] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:04.523455 6033 tablet_bootstrap.cc:492] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:04.523856 6072 log.cc:1196] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/f0a957251a6540c9b4a83a3517eb4ed6
I20260808 13:35:04.524938 6072 ts_tablet_manager.cc:1939] T f0a957251a6540c9b4a83a3517eb4ed6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:04.529116 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet f0a957251a6540c9b4a83a3517eb4ed6 (table default.Table_a297438a46d34ceaac3836c97edd6f38 [id=462ae7e5fbb8461ebd32fa4cd72c941e]) successfully deleted
I20260808 13:35:04.529911 6033 tablet_bootstrap.cc:654] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:04.530305 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet ae5487d6a4204a6f8b97a1d09d1314ab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae5487d6a4204a6f8b97a1d09d1314ab already in progress: creating tablet
I20260808 13:35:04.543919 6074 ts_tablet_manager.cc:1918] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:04.544525 6074 log.cc:1196] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/f0a957251a6540c9b4a83a3517eb4ed6
I20260808 13:35:04.545761 6074 ts_tablet_manager.cc:1939] T f0a957251a6540c9b4a83a3517eb4ed6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:04.551620 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet f0a957251a6540c9b4a83a3517eb4ed6 (table default.Table_a297438a46d34ceaac3836c97edd6f38 [id=462ae7e5fbb8461ebd32fa4cd72c941e]) successfully deleted
W20260808 13:35:04.552929 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet ae5487d6a4204a6f8b97a1d09d1314ab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae5487d6a4204a6f8b97a1d09d1314ab already in progress: creating tablet
I20260808 13:35:04.555335 6075 ts_tablet_manager.cc:1918] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:04.555828 6075 log.cc:1196] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/f0a957251a6540c9b4a83a3517eb4ed6
I20260808 13:35:04.556209 6046 tablet_bootstrap.cc:654] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.560230 6075 ts_tablet_manager.cc:1939] T f0a957251a6540c9b4a83a3517eb4ed6 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:04.564046 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet f0a957251a6540c9b4a83a3517eb4ed6 (table default.Table_a297438a46d34ceaac3836c97edd6f38 [id=462ae7e5fbb8461ebd32fa4cd72c941e]) successfully deleted
I20260808 13:35:04.566133 6046 tablet_bootstrap.cc:492] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:04.566536 6046 ts_tablet_manager.cc:1397] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.065s user 0.010s sys 0.002s
I20260808 13:35:04.568953 6046 raft_consensus.cc:348] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:04.570852 6046 raft_consensus.cc:374] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.571158 6046 raft_consensus.cc:729] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.571792 6033 tablet_bootstrap.cc:492] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:04.572343 6033 ts_tablet_manager.cc:1397] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.049s user 0.012s sys 0.001s
I20260808 13:35:04.572762 6046 consensus_queue.cc:260] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:04.579146 6046 ts_tablet_manager.cc:1428] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260808 13:35:04.585371 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1ccce545a53c4f4ca79ede2bee9ebceb" } modify_external_catalogs: true
I20260808 13:35:04.585963 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1ccce545a53c4f4ca79ede2bee9ebceb" } modify_external_catalogs: true
I20260808 13:35:04.576620 6033 raft_consensus.cc:348] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:04.596359 6033 raft_consensus.cc:374] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.596781 6033 raft_consensus.cc:729] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.597668 6033 consensus_queue.cc:260] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:04.612787 6033 ts_tablet_manager.cc:1428] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260808 13:35:04.666734 5615 tablet_service.cc:1467] Processing CreateTablet for tablet a1a5a9f860b34797825be7568a0b4ccb (DEFAULT_TABLE table=default.Table_d83818a7ac294a0cbe2f914264d02c59 [id=807c9b982b81419e903344d491fabe09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.666848 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 38aab2b333884120b1851c8cdf93d140 (table default.Table_1ccce545a53c4f4ca79ede2bee9ebceb [id=7d8eeac8f7374777a4be8b441045af2d]). Expected version 1 got 0
I20260808 13:35:04.668386 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a5a9f860b34797825be7568a0b4ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.674794 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 38aab2b333884120b1851c8cdf93d140 (table default.Table_1ccce545a53c4f4ca79ede2bee9ebceb [id=7d8eeac8f7374777a4be8b441045af2d]). Expected version 1 got 0
I20260808 13:35:04.681243 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_7cd941437b494bdba3bc98e7bfad0c78" } new_table_name: "default.Table_b034596e0aa1406ab5a3a849c74a03cc" modify_external_catalogs: true
W20260808 13:35:04.688007 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5f18e5b0a4c445a88b7717441edb323f (table default.Table_e79e02d050604203b805744df0be2c8c [id=3540defcda954120ba76fe9e4c3869a8], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5f18e5b0a4c445a88b7717441edb323f consensus state has no leader
I20260808 13:35:04.688922 5357 tablet_service.cc:1467] Processing CreateTablet for tablet a1a5a9f860b34797825be7568a0b4ccb (DEFAULT_TABLE table=default.Table_d83818a7ac294a0cbe2f914264d02c59 [id=807c9b982b81419e903344d491fabe09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.689224 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5575e54961db47308b8acdcac7b5431d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:04.690088 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a5a9f860b34797825be7568a0b4ccb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:04.693173 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1a5a9f860b34797825be7568a0b4ccb consensus state has no leader
I20260808 13:35:04.699599 5844 tablet_service.cc:1467] Processing CreateTablet for tablet a1a5a9f860b34797825be7568a0b4ccb (DEFAULT_TABLE table=default.Table_18666f1bccbb4a41ba39aafa9354326b [id=807c9b982b81419e903344d491fabe09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.701196 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a5a9f860b34797825be7568a0b4ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.703245 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 5aaf4fe334334b5db2fcc1bc8992c113 (DEFAULT_TABLE table=default.Table_c08ec3a916d44f25b61fa57b88efcbcc [id=3d42b09e05b3458caffce3214d58f93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.704684 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aaf4fe334334b5db2fcc1bc8992c113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.704535 5355 tablet_service.cc:1467] Processing CreateTablet for tablet c59e1f5f36a2480bbb7288a1449a54de (DEFAULT_TABLE table=default.Table_5f0e676e2cf14d31b0439a4fdd39a58f [id=9a99bed19d724319bfaae469dca0ed62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.706013 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59e1f5f36a2480bbb7288a1449a54de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.709139 5845 tablet_service.cc:1467] Processing CreateTablet for tablet c59e1f5f36a2480bbb7288a1449a54de (DEFAULT_TABLE table=default.Table_5f0e676e2cf14d31b0439a4fdd39a58f [id=9a99bed19d724319bfaae469dca0ed62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.711053 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59e1f5f36a2480bbb7288a1449a54de. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:04.715489 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5575e54961db47308b8acdcac7b5431d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:04.720343 5615 tablet_service.cc:1467] Processing CreateTablet for tablet c59e1f5f36a2480bbb7288a1449a54de (DEFAULT_TABLE table=default.Table_5f0e676e2cf14d31b0439a4fdd39a58f [id=9a99bed19d724319bfaae469dca0ed62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.723723 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59e1f5f36a2480bbb7288a1449a54de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.745973 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 5aaf4fe334334b5db2fcc1bc8992c113 (DEFAULT_TABLE table=default.Table_c08ec3a916d44f25b61fa57b88efcbcc [id=3d42b09e05b3458caffce3214d58f93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.747490 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aaf4fe334334b5db2fcc1bc8992c113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.753664 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 5aaf4fe334334b5db2fcc1bc8992c113 (DEFAULT_TABLE table=default.Table_c08ec3a916d44f25b61fa57b88efcbcc [id=3d42b09e05b3458caffce3214d58f93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.756047 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 106aef8e805d4a0ebaaf2bdd808bbfab (DEFAULT_TABLE table=default.Table_c64b1240bf04403d83f297ac2d40590c [id=984c64fad49d47aca100cfe561c0b780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.759280 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 106aef8e805d4a0ebaaf2bdd808bbfab (DEFAULT_TABLE table=default.Table_c64b1240bf04403d83f297ac2d40590c [id=984c64fad49d47aca100cfe561c0b780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.760775 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106aef8e805d4a0ebaaf2bdd808bbfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.763458 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106aef8e805d4a0ebaaf2bdd808bbfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.767306 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aaf4fe334334b5db2fcc1bc8992c113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.775542 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 106aef8e805d4a0ebaaf2bdd808bbfab (DEFAULT_TABLE table=default.Table_c64b1240bf04403d83f297ac2d40590c [id=984c64fad49d47aca100cfe561c0b780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.777036 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106aef8e805d4a0ebaaf2bdd808bbfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.779312 6026 tablet_bootstrap.cc:492] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:04.779759 6026 ts_tablet_manager.cc:1397] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.275s user 0.082s sys 0.139s
I20260808 13:35:04.783030 6026 raft_consensus.cc:348] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:04.783749 6026 raft_consensus.cc:374] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.784049 6026 raft_consensus.cc:729] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.790920 6026 consensus_queue.cc:260] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:04.794936 6026 ts_tablet_manager.cc:1428] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
W20260808 13:35:04.796761 4837 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet edfa66acceea4b54911d78ee16609ef7 has no consensus state
I20260808 13:35:04.797410 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 5aaf4fe334334b5db2fcc1bc8992c113
I20260808 13:35:04.800218 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 38aab2b333884120b1851c8cdf93d140
I20260808 13:35:04.800195 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aaf4fe334334b5db2fcc1bc8992c113 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:04.802241 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 5aaf4fe334334b5db2fcc1bc8992c113 because the tablet was not found. No further retry: Not found: Tablet not found: 5aaf4fe334334b5db2fcc1bc8992c113
I20260808 13:35:04.803507 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 38aab2b333884120b1851c8cdf93d140 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:04.804275 6074 tablet_replica.cc:330] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:04.805037 6074 raft_consensus.cc:2229] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:04.805496 6074 raft_consensus.cc:2258] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:04.807897 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 38aab2b333884120b1851c8cdf93d140 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:04.808895 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aaf4fe334334b5db2fcc1bc8992c113 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:04.811323 6074 ts_tablet_manager.cc:1905] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:04.816015 5353 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aaf4fe334334b5db2fcc1bc8992c113 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:04.817229 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 38aab2b333884120b1851c8cdf93d140 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:04.818457 6075 tablet_replica.cc:330] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:04.819237 6075 raft_consensus.cc:2229] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:04.819686 6075 raft_consensus.cc:2258] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:04.820231 6072 tablet_replica.cc:330] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:04.820694 6026 tablet_bootstrap.cc:492] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:04.820972 6072 raft_consensus.cc:2229] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:04.821415 6072 raft_consensus.cc:2258] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:04.823319 6072 ts_tablet_manager.cc:1905] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:04.827852 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 5aaf4fe334334b5db2fcc1bc8992c113 because the tablet was not found. No further retry: Not found: Tablet not found: 5aaf4fe334334b5db2fcc1bc8992c113
I20260808 13:35:04.832142 6026 tablet_bootstrap.cc:654] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.830309 6075 ts_tablet_manager.cc:1905] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:04.836308 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c3030f653d7b4808b84d7dbc2c938d79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:04.838017 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c3030f653d7b4808b84d7dbc2c938d79 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:04.841575 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5dcf7a0821714a82a1888b343d9c6678"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:04.843381 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5dcf7a0821714a82a1888b343d9c6678 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:04.851218 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet ae5487d6a4204a6f8b97a1d09d1314ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:04.857053 6072 ts_tablet_manager.cc:1918] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:04.857460 6072 log.cc:1196] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/38aab2b333884120b1851c8cdf93d140
I20260808 13:35:04.858178 6072 ts_tablet_manager.cc:1939] T 38aab2b333884120b1851c8cdf93d140 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:04.871556 6046 tablet_bootstrap.cc:492] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:04.873397 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 38aab2b333884120b1851c8cdf93d140 (table default.Table_1ccce545a53c4f4ca79ede2bee9ebceb [id=7d8eeac8f7374777a4be8b441045af2d]) successfully deleted
W20260808 13:35:04.877179 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 5aaf4fe334334b5db2fcc1bc8992c113 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5aaf4fe334334b5db2fcc1bc8992c113 already in progress: creating tablet
I20260808 13:35:04.880805 6026 tablet_bootstrap.cc:492] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:04.881268 6026 ts_tablet_manager.cc:1397] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.061s user 0.010s sys 0.000s
I20260808 13:35:04.883914 6026 raft_consensus.cc:348] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.884639 6026 raft_consensus.cc:374] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.884943 6026 raft_consensus.cc:729] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.885602 6026 consensus_queue.cc:260] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.888450 5615 tablet_service.cc:1467] Processing CreateTablet for tablet 8948ce98e7a941f59fa9932fe241638d (DEFAULT_TABLE table=default.Table_feaae5f9ed9e4419851beead826e589f [id=c4338f1c19ec4210a803a1e2aaaa65d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.889560 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8948ce98e7a941f59fa9932fe241638d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.901373 6072 tablet_replica.cc:330] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:04.905131 6026 ts_tablet_manager.cc:1428] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.024s user 0.008s sys 0.001s
I20260808 13:35:04.907969 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet ae5487d6a4204a6f8b97a1d09d1314ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:04.912596 6072 raft_consensus.cc:2229] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:04.912223 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 3e36f7753d9d4c43afc3c9b253a03846 (DEFAULT_TABLE table=default.Table_e4d8738a572f475a9becc0e4e90d6c1a [id=e794d9029f6446a996ff766936695c20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.915724 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e36f7753d9d4c43afc3c9b253a03846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.927942 6046 tablet_bootstrap.cc:654] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.929580 6026 tablet_bootstrap.cc:492] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:04.932056 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet a1a5a9f860b34797825be7568a0b4ccb (table default.Table_18666f1bccbb4a41ba39aafa9354326b [id=807c9b982b81419e903344d491fabe09]). Expected version 2 got 0
I20260808 13:35:04.933053 6033 tablet_bootstrap.cc:492] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:04.905866 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 8948ce98e7a941f59fa9932fe241638d (DEFAULT_TABLE table=default.Table_feaae5f9ed9e4419851beead826e589f [id=c4338f1c19ec4210a803a1e2aaaa65d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.935531 6026 tablet_bootstrap.cc:654] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.935611 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8948ce98e7a941f59fa9932fe241638d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.938975 6033 tablet_bootstrap.cc:654] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.942476 4840 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c3030f653d7b4808b84d7dbc2c938d79" } new_table_name: "default.Table_fd7c3f5b3751428695967c6657d567d6" modify_external_catalogs: true
I20260808 13:35:04.954937 5615 tablet_service.cc:1467] Processing CreateTablet for tablet edfa66acceea4b54911d78ee16609ef7 (DEFAULT_TABLE table=default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.956167 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edfa66acceea4b54911d78ee16609ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.962077 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet a1a5a9f860b34797825be7568a0b4ccb (table default.Table_18666f1bccbb4a41ba39aafa9354326b [id=807c9b982b81419e903344d491fabe09]). Expected version 2 got 0
I20260808 13:35:04.913270 6072 raft_consensus.cc:2258] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:04.970752 6072 ts_tablet_manager.cc:1905] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:04.976138 6026 tablet_bootstrap.cc:492] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:04.976863 5346 tablet_service.cc:1514] Processing DeleteTablet for tablet ae5487d6a4204a6f8b97a1d09d1314ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:04.977980 6033 tablet_bootstrap.cc:492] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:04.978405 6033 ts_tablet_manager.cc:1397] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.046s user 0.006s sys 0.004s
I20260808 13:35:04.977108 6026 ts_tablet_manager.cc:1397] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.048s user 0.010s sys 0.018s
I20260808 13:35:04.978107 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 3e36f7753d9d4c43afc3c9b253a03846 (DEFAULT_TABLE table=default.Table_e4d8738a572f475a9becc0e4e90d6c1a [id=e794d9029f6446a996ff766936695c20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:04.979658 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e36f7753d9d4c43afc3c9b253a03846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.981016 6026 raft_consensus.cc:348] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.981798 6026 raft_consensus.cc:374] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.982129 6026 raft_consensus.cc:729] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.982445 6033 raft_consensus.cc:348] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.983210 6033 raft_consensus.cc:374] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:04.982836 6026 consensus_queue.cc:260] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.983522 6033 raft_consensus.cc:729] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:04.985018 6074 ts_tablet_manager.cc:1918] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:04.985435 6026 ts_tablet_manager.cc:1428] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 13:35:04.985440 6074 log.cc:1196] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/38aab2b333884120b1851c8cdf93d140
I20260808 13:35:04.986133 6074 ts_tablet_manager.cc:1939] T 38aab2b333884120b1851c8cdf93d140 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:04.988909 6074 tablet_replica.cc:330] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:04.989647 6074 raft_consensus.cc:2229] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:04.990095 6074 raft_consensus.cc:2258] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:04.990075 6069 raft_consensus.cc:482] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:04.990654 6069 raft_consensus.cc:504] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:04.991343 6026 tablet_bootstrap.cc:492] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:04.995600 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 38aab2b333884120b1851c8cdf93d140 (table default.Table_1ccce545a53c4f4ca79ede2bee9ebceb [id=7d8eeac8f7374777a4be8b441045af2d]) successfully deleted
I20260808 13:35:04.995184 5345 tablet_service.cc:1467] Processing CreateTablet for tablet edfa66acceea4b54911d78ee16609ef7 (DEFAULT_TABLE table=default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.005599 6026 tablet_bootstrap.cc:654] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:04.982821 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 8948ce98e7a941f59fa9932fe241638d (DEFAULT_TABLE table=default.Table_feaae5f9ed9e4419851beead826e589f [id=c4338f1c19ec4210a803a1e2aaaa65d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.009155 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edfa66acceea4b54911d78ee16609ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:05.001852 4838 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_feaae5f9ed9e4419851beead826e589f" } new_table_name: "default.Table_5eabc9a0e2b74737ac3e20af0fa63336" modify_external_catalogs: true
I20260808 13:35:05.008412 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5575e54961db47308b8acdcac7b5431d" } modify_external_catalogs: true
I20260808 13:35:05.011018 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5575e54961db47308b8acdcac7b5431d" } modify_external_catalogs: true
I20260808 13:35:04.994097 6069 leader_election.cc:288] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:05.017037 6072 ts_tablet_manager.cc:1918] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:05.019918 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_aa32f9d1cc2c458a8f439cd65ca098e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:05.025257 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa32f9d1cc2c458a8f439cd65ca098e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:05.026755 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8948ce98e7a941f59fa9932fe241638d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:05.029150 5844 tablet_service.cc:1467] Processing CreateTablet for tablet edfa66acceea4b54911d78ee16609ef7 (DEFAULT_TABLE table=default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.017571 6072 log.cc:1196] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/ae5487d6a4204a6f8b97a1d09d1314ab
I20260808 13:35:05.035488 6072 ts_tablet_manager.cc:1939] T ae5487d6a4204a6f8b97a1d09d1314ab P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:05.027387 4839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_06f4a6ab47814783a9277ded39511f8f" } modify_external_catalogs: true
I20260808 13:35:05.036976 4839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_06f4a6ab47814783a9277ded39511f8f" } modify_external_catalogs: true
I20260808 13:35:05.040155 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edfa66acceea4b54911d78ee16609ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:05.038373 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 3e36f7753d9d4c43afc3c9b253a03846 (DEFAULT_TABLE table=default.Table_e4d8738a572f475a9becc0e4e90d6c1a [id=e794d9029f6446a996ff766936695c20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.049227 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e36f7753d9d4c43afc3c9b253a03846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:04.992775 6033 consensus_queue.cc:260] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.037964 6074 ts_tablet_manager.cc:1905] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:05.051842 6039 raft_consensus.cc:482] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:05.052652 6039 raft_consensus.cc:504] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.069212 6039 leader_election.cc:288] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:05.070674 4841 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet a87c01c44b114c4cb8e62c32222ed807
I20260808 13:35:05.082099 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_41ac7940bf8641a0af3a6117b13430cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:05.084400 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41ac7940bf8641a0af3a6117b13430cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:05.101550 4838 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8948ce98e7a941f59fa9932fe241638d consensus state has no leader
W20260808 13:35:05.102953 4840 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 953a2a65dea7468c8356931cabb6bfc9 has no consensus state
I20260808 13:35:05.103705 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_41aa30b0f19f470ea3c1f10a3bf72c73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:05.105438 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41aa30b0f19f470ea3c1f10a3bf72c73 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:05.111406 6075 ts_tablet_manager.cc:1918] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:05.111832 6075 log.cc:1196] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/38aab2b333884120b1851c8cdf93d140
I20260808 13:35:05.125180 6075 ts_tablet_manager.cc:1939] T 38aab2b333884120b1851c8cdf93d140 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:05.127555 6075 tablet_replica.cc:330] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:05.128239 6075 raft_consensus.cc:2229] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:05.128687 6075 raft_consensus.cc:2258] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:05.130365 6075 ts_tablet_manager.cc:1905] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:05.132457 6046 tablet_bootstrap.cc:492] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:05.132890 6046 ts_tablet_manager.cc:1397] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.262s user 0.003s sys 0.012s
I20260808 13:35:05.136240 6033 ts_tablet_manager.cc:1428] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.158s user 0.007s sys 0.001s
I20260808 13:35:05.139528 6033 tablet_bootstrap.cc:492] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:05.141490 6103 raft_consensus.cc:482] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:05.150898 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 38aab2b333884120b1851c8cdf93d140 (table default.Table_1ccce545a53c4f4ca79ede2bee9ebceb [id=7d8eeac8f7374777a4be8b441045af2d]) successfully deleted
I20260808 13:35:05.142992 6103 raft_consensus.cc:504] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.184113 6103 leader_election.cc:288] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:05.191804 6046 raft_consensus.cc:348] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.192942 6046 raft_consensus.cc:374] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:05.193406 6046 raft_consensus.cc:729] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:05.195235 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_ea98c448b3f3473683c7e7e0ac5ae2be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:05.198066 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea98c448b3f3473683c7e7e0ac5ae2be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:05.199374 6074 ts_tablet_manager.cc:1918] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:05.209717 6074 log.cc:1196] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/ae5487d6a4204a6f8b97a1d09d1314ab
I20260808 13:35:05.213166 6046 consensus_queue.cc:260] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.221604 6046 ts_tablet_manager.cc:1428] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.088s user 0.007s sys 0.004s
I20260808 13:35:05.227468 6074 ts_tablet_manager.cc:1939] T ae5487d6a4204a6f8b97a1d09d1314ab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:05.227865 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet ae5487d6a4204a6f8b97a1d09d1314ab (table default.Table_1ab91b8bd91d4fb9af1ffc1e9c6c7c5a [id=490c4641a4554492a415f6b987c72300]) successfully deleted
I20260808 13:35:05.230031 6052 raft_consensus.cc:482] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:05.230650 6052 raft_consensus.cc:504] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.234658 6052 leader_election.cc:288] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:05.235214 6052 raft_consensus.cc:482] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:05.235731 6052 raft_consensus.cc:504] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.237994 6075 ts_tablet_manager.cc:1918] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:05.238591 6075 log.cc:1196] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/ae5487d6a4204a6f8b97a1d09d1314ab
I20260808 13:35:05.239329 6052 leader_election.cc:288] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:05.239794 6075 ts_tablet_manager.cc:1939] T ae5487d6a4204a6f8b97a1d09d1314ab P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:05.252024 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet ae5487d6a4204a6f8b97a1d09d1314ab (table default.Table_1ab91b8bd91d4fb9af1ffc1e9c6c7c5a [id=490c4641a4554492a415f6b987c72300]) successfully deleted
I20260808 13:35:05.266402 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f18e5b0a4c445a88b7717441edb323f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:05.267499 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106aef8e805d4a0ebaaf2bdd808bbfab" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:05.268095 5385 raft_consensus.cc:2454] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:05.285099 6046 tablet_bootstrap.cc:492] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:05.335929 4837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5f0e676e2cf14d31b0439a4fdd39a58f" } modify_external_catalogs: true
I20260808 13:35:05.337275 4837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5f0e676e2cf14d31b0439a4fdd39a58f" } modify_external_catalogs: true
I20260808 13:35:05.340946 4832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_aa32f9d1cc2c458a8f439cd65ca098e0" } new_table_name: "default.Table_60e5f930145b4e7db14190e5896be3f4" modify_external_catalogs: true
I20260808 13:35:05.333554 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5dcf7a0821714a82a1888b343d9c6678" } new_table_name: "default.Table_7e2a1be30ada4555aef1f6abf581ccfc" modify_external_catalogs: true
I20260808 13:35:05.382014 6033 tablet_bootstrap.cc:654] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:05.403959 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f18e5b0a4c445a88b7717441edb323f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:05.405153 5644 raft_consensus.cc:2454] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:05.407840 5285 leader_election.cc:302] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:05.409114 6069 raft_consensus.cc:2781] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:05.409545 6069 raft_consensus.cc:482] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:05.410030 6069 raft_consensus.cc:3037] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:05.411877 5551 leader_election.cc:341] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:05.418287 5551 leader_election.cc:302] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:05.420450 6111 raft_consensus.cc:2781] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:05.421094 6111 raft_consensus.cc:482] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:05.421182 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet ae5487d6a4204a6f8b97a1d09d1314ab (table default.Table_1ab91b8bd91d4fb9af1ffc1e9c6c7c5a [id=490c4641a4554492a415f6b987c72300]) successfully deleted
I20260808 13:35:05.421808 6111 raft_consensus.cc:3037] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:05.429401 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106aef8e805d4a0ebaaf2bdd808bbfab" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:05.432085 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f18e5b0a4c445a88b7717441edb323f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:05.438445 5883 raft_consensus.cc:2454] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:05.439699 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_de035120cda84e3da95340a3967f4396"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:05.440186 5882 raft_consensus.cc:2454] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
W20260808 13:35:05.445402 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de035120cda84e3da95340a3967f4396 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:05.446832 6026 tablet_bootstrap.cc:492] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:05.447440 6026 ts_tablet_manager.cc:1397] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.456s user 0.067s sys 0.103s
I20260808 13:35:05.452538 6026 raft_consensus.cc:348] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.453460 6026 raft_consensus.cc:374] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:05.453802 6026 raft_consensus.cc:729] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:05.455623 5550 leader_election.cc:302] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:05.459769 6052 raft_consensus.cc:2781] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:05.460426 6052 raft_consensus.cc:482] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:05.454439 6026 consensus_queue.cc:260] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.461117 6052 raft_consensus.cc:3037] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:05.471220 6069 raft_consensus.cc:504] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.454639 6111 raft_consensus.cc:504] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.478732 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f18e5b0a4c445a88b7717441edb323f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:05.480244 5644 raft_consensus.cc:2379] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:05.481244 6111 leader_election.cc:288] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:05.483219 6026 ts_tablet_manager.cc:1428] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.035s user 0.011s sys 0.012s
I20260808 13:35:05.482982 6069 leader_election.cc:288] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:05.484275 6026 tablet_bootstrap.cc:492] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:05.487089 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f18e5b0a4c445a88b7717441edb323f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:05.488817 5385 raft_consensus.cc:2379] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:05.494985 6052 raft_consensus.cc:504] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.496493 6026 tablet_bootstrap.cc:654] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:05.503237 6046 tablet_bootstrap.cc:654] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:05.512459 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106aef8e805d4a0ebaaf2bdd808bbfab" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:05.513720 6052 leader_election.cc:288] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:05.514595 5551 leader_election.cc:341] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:05.525305 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106aef8e805d4a0ebaaf2bdd808bbfab" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:05.526206 5883 raft_consensus.cc:3037] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:05.522868 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f18e5b0a4c445a88b7717441edb323f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:05.532332 5882 raft_consensus.cc:3037] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:05.550058 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106aef8e805d4a0ebaaf2bdd808bbfab" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:05.552304 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f18e5b0a4c445a88b7717441edb323f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:05.553673 5386 raft_consensus.cc:2379] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:05.553655 6046 tablet_bootstrap.cc:492] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:05.558744 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f18e5b0a4c445a88b7717441edb323f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:05.559329 6046 ts_tablet_manager.cc:1397] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.275s user 0.005s sys 0.023s
I20260808 13:35:05.560307 5642 raft_consensus.cc:2379] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:05.562155 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106aef8e805d4a0ebaaf2bdd808bbfab" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:05.563963 5644 raft_consensus.cc:2379] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:05.566380 6046 raft_consensus.cc:348] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.567590 6046 raft_consensus.cc:374] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:05.568013 6046 raft_consensus.cc:729] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:05.568984 6046 consensus_queue.cc:260] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.582743 6046 ts_tablet_manager.cc:1428] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.023s user 0.007s sys 0.005s
I20260808 13:35:05.584460 6046 tablet_bootstrap.cc:492] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:05.595228 5771 leader_election.cc:341] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:05.603411 5772 leader_election.cc:302] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:05.606228 5772 leader_election.cc:302] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:05.633313 5883 raft_consensus.cc:2454] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:05.635068 5550 leader_election.cc:302] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:05.636343 6052 raft_consensus.cc:2781] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:05.636883 6052 raft_consensus.cc:686] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:05.637652 6046 tablet_bootstrap.cc:654] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:05.637898 6052 consensus_queue.cc:237] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.645767 6069 raft_consensus.cc:482] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:05.646488 6069 raft_consensus.cc:504] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.652451 6069 leader_election.cc:288] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:05.658512 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_488d26ecb7bd4a83a08f3722b9297990"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:05.660550 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_488d26ecb7bd4a83a08f3722b9297990 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:05.649070 6120 raft_consensus.cc:2726] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:05.666066 4832 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e2a95712de254c13b3b6dc4fe5cb15f7 has no consensus state
W20260808 13:35:05.671809 4833 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75740d7986e0439a8677dc14d29ad0e5 has no consensus state
I20260808 13:35:05.678943 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a5a9f860b34797825be7568a0b4ccb" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:05.679975 5644 raft_consensus.cc:2454] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:05.680147 6026 tablet_bootstrap.cc:492] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:05.680677 6026 ts_tablet_manager.cc:1397] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.197s user 0.019s sys 0.028s
I20260808 13:35:05.684016 6026 raft_consensus.cc:348] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.684839 6026 raft_consensus.cc:374] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:05.685113 6026 raft_consensus.cc:729] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:05.686379 6026 consensus_queue.cc:260] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.689364 6026 ts_tablet_manager.cc:1428] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 13:35:05.691912 5285 leader_election.cc:302] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:05.693348 6069 raft_consensus.cc:2781] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:05.693727 6069 raft_consensus.cc:482] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:05.694150 6069 raft_consensus.cc:3037] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:05.714898 4821 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 75740d7986e0439a8677dc14d29ad0e5 (table default.Table_7e2a1be30ada4555aef1f6abf581ccfc [id=fdacc3a340d44e89a022fd9311d68fcc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 75740d7986e0439a8677dc14d29ad0e5 has no consensus state
I20260808 13:35:05.713660 6069 raft_consensus.cc:504] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.724880 6069 leader_election.cc:288] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:05.747758 5882 raft_consensus.cc:2454] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
W20260808 13:35:05.749194 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 143089us
I20260808 13:35:05.752655 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f18e5b0a4c445a88b7717441edb323f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:05.752595 6103 raft_consensus.cc:2726] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:05.754454 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a5a9f860b34797825be7568a0b4ccb" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:05.757719 5881 raft_consensus.cc:2379] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:05.764094 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_3009d485300e4bc78022312d9ddd9a34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:05.766858 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a5a9f860b34797825be7568a0b4ccb" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:05.768281 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3009d485300e4bc78022312d9ddd9a34 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:05.769740 5283 leader_election.cc:341] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:05.773027 5283 leader_election.cc:302] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:05.775272 6069 raft_consensus.cc:2726] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 13:35:05.787703 6046 tablet_bootstrap.cc:492] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:05.788741 6046 ts_tablet_manager.cc:1397] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.205s user 0.025s sys 0.037s
I20260808 13:35:05.791949 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_414cb3a438f24b8181e2d5d53bd1f3e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:05.795295 6033 tablet_bootstrap.cc:492] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:05.795946 6033 ts_tablet_manager.cc:1397] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.657s user 0.014s sys 0.011s
W20260808 13:35:05.795675 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_414cb3a438f24b8181e2d5d53bd1f3e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:05.807111 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f18e5b0a4c445a88b7717441edb323f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:05.807889 5882 raft_consensus.cc:2379] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
W20260808 13:35:05.809283 5283 leader_election.cc:341] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:05.810863 6033 raft_consensus.cc:348] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.813120 6033 raft_consensus.cc:374] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:05.813599 6033 raft_consensus.cc:729] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:05.813513 6046 raft_consensus.cc:348] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.814570 6033 consensus_queue.cc:260] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.815564 6046 raft_consensus.cc:374] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260808 13:35:05.817166 4836 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 17) took 797 ms (client timeout 1000 ms). Trace:
I20260808 13:35:05.820900 6046 raft_consensus.cc:729] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:05.821614 4840 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet a1a5a9f860b34797825be7568a0b4ccb (table default.Table_18666f1bccbb4a41ba39aafa9354326b [id=807c9b982b81419e903344d491fabe09]). Expected version 2 got 0
I20260808 13:35:05.823402 6033 ts_tablet_manager.cc:1428] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.027s user 0.006s sys 0.003s
I20260808 13:35:05.824441 6033 tablet_bootstrap.cc:492] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:05.830687 6033 tablet_bootstrap.cc:654] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:05.831264 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 75740d7986e0439a8677dc14d29ad0e5 (DEFAULT_TABLE table=default.Table_7e2a1be30ada4555aef1f6abf581ccfc [id=fdacc3a340d44e89a022fd9311d68fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.832481 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75740d7986e0439a8677dc14d29ad0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:05.835496 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 1c140276a5d7438b8c63e02daecd3f80 (DEFAULT_TABLE table=default.Table_06f4a6ab47814783a9277ded39511f8f [id=07e09f6780de40928b85f84a463936bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.836624 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c140276a5d7438b8c63e02daecd3f80. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:05.838658 4836 rpcz_store.cc:269] 0808 13:35:05.019452 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:05.023197 (+ 3745us) service_pool.cc:227] Handling call
0808 13:35:05.065199 (+ 42002us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:05.817085 (+751886us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":31673088}
I20260808 13:35:05.821648 6046 consensus_queue.cc:260] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.845348 6046 ts_tablet_manager.cc:1428] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.040s user 0.007s sys 0.000s
I20260808 13:35:05.853229 6033 tablet_bootstrap.cc:492] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:05.853906 6033 ts_tablet_manager.cc:1397] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.000s
I20260808 13:35:05.857455 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 75740d7986e0439a8677dc14d29ad0e5 (DEFAULT_TABLE table=default.Table_7e2a1be30ada4555aef1f6abf581ccfc [id=fdacc3a340d44e89a022fd9311d68fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.857285 6033 raft_consensus.cc:348] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.858292 6033 raft_consensus.cc:374] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:05.858741 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75740d7986e0439a8677dc14d29ad0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:05.863545 5345 tablet_service.cc:1467] Processing CreateTablet for tablet 1c140276a5d7438b8c63e02daecd3f80 (DEFAULT_TABLE table=default.Table_06f4a6ab47814783a9277ded39511f8f [id=07e09f6780de40928b85f84a463936bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.864796 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c140276a5d7438b8c63e02daecd3f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:05.866011 6033 raft_consensus.cc:729] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:05.867172 6033 consensus_queue.cc:260] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:05.869421 6033 ts_tablet_manager.cc:1428] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20260808 13:35:05.879851 4835 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_18666f1bccbb4a41ba39aafa9354326b" } new_table_name: "default.Table_34602dc423cc4b8b8cc125d6c1ce5452" modify_external_catalogs: true
I20260808 13:35:05.881434 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 953a2a65dea7468c8356931cabb6bfc9 (DEFAULT_TABLE table=default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.883558 6046 tablet_bootstrap.cc:492] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:05.884095 6033 tablet_bootstrap.cc:492] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:05.896617 5550 leader_election.cc:302] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:05.896808 4840 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_41ac7940bf8641a0af3a6117b13430cb" } new_table_name: "default.Table_40a75c335146479fb7dc85888769c778" modify_external_catalogs: true
I20260808 13:35:05.899588 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 953a2a65dea7468c8356931cabb6bfc9 (DEFAULT_TABLE table=default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.901796 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 953a2a65dea7468c8356931cabb6bfc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:05.915591 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 953a2a65dea7468c8356931cabb6bfc9 (DEFAULT_TABLE table=default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.956647 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 953a2a65dea7468c8356931cabb6bfc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:05.996510 6046 tablet_bootstrap.cc:654] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:06.001987 5615 tablet_service.cc:1467] Processing CreateTablet for tablet 75740d7986e0439a8677dc14d29ad0e5 (DEFAULT_TABLE table=default.Table_7e2a1be30ada4555aef1f6abf581ccfc [id=fdacc3a340d44e89a022fd9311d68fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:05.730374 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a5a9f860b34797825be7568a0b4ccb" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:06.008025 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 1c140276a5d7438b8c63e02daecd3f80 (DEFAULT_TABLE table=default.Table_06f4a6ab47814783a9277ded39511f8f [id=07e09f6780de40928b85f84a463936bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.009902 5644 raft_consensus.cc:3037] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:06.011852 6103 raft_consensus.cc:482] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:06.012706 6103 raft_consensus.cc:504] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.020141 6103 leader_election.cc:288] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:06.020798 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aaf4fe334334b5db2fcc1bc8992c113" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:06.023626 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aaf4fe334334b5db2fcc1bc8992c113" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:06.024595 5386 raft_consensus.cc:2454] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:06.022454 5642 raft_consensus.cc:2454] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:06.026543 5771 leader_election.cc:302] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:06.027752 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 953a2a65dea7468c8356931cabb6bfc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.029305 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75740d7986e0439a8677dc14d29ad0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.031603 6033 tablet_bootstrap.cc:654] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:06.035274 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c140276a5d7438b8c63e02daecd3f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.041493 6103 raft_consensus.cc:2781] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:06.045630 6103 raft_consensus.cc:482] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:06.046173 6103 raft_consensus.cc:3037] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:06.061252 4837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet c59e1f5f36a2480bbb7288a1449a54de
I20260808 13:35:06.066548 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet c59e1f5f36a2480bbb7288a1449a54de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:06.075714 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet c59e1f5f36a2480bbb7288a1449a54de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:06.079145 5845 tablet_service.cc:1514] Processing DeleteTablet for tablet c59e1f5f36a2480bbb7288a1449a54de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:05.690449 6026 tablet_bootstrap.cc:492] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:06.102916 6114 raft_consensus.cc:2781] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:06.107551 6033 tablet_bootstrap.cc:492] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:06.108076 6033 ts_tablet_manager.cc:1397] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.224s user 0.011s sys 0.011s
I20260808 13:35:06.135025 6114 raft_consensus.cc:686] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:06.140434 6114 consensus_queue.cc:237] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.133880 6033 raft_consensus.cc:348] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.161872 6033 raft_consensus.cc:374] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:06.165205 6103 raft_consensus.cc:504] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.166432 6033 raft_consensus.cc:729] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:06.167443 6033 consensus_queue.cc:260] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.170133 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 1c140276a5d7438b8c63e02daecd3f80
I20260808 13:35:06.170409 6103 leader_election.cc:288] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:06.183686 4839 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 36) took 1156 ms. Trace:
I20260808 13:35:06.185600 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_763498da5cad48e88d04c14346559749"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:06.190425 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_763498da5cad48e88d04c14346559749 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:06.192301 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aaf4fe334334b5db2fcc1bc8992c113" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:06.192502 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c140276a5d7438b8c63e02daecd3f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:06.193447 5386 raft_consensus.cc:3037] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:06.212059 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c140276a5d7438b8c63e02daecd3f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:06.214280 6033 ts_tablet_manager.cc:1428] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.106s user 0.011s sys 0.000s
I20260808 13:35:06.194133 4839 rpcz_store.cc:276] 0808 13:35:05.026903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:05.027043 (+ 140us) service_pool.cc:227] Handling call
0808 13:35:05.037524 (+ 10481us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:05.063675 (+ 26151us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:05.063808 (+ 133us) catalog_manager.cc:2761] Locking tablets
0808 13:35:05.818569 (+754761us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:06.165079 (+346510us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:06.165641 (+ 562us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:06.165953 (+ 312us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:06.183546 (+ 17593us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:05.836044 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:05.836446 (+ 402us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:05.836476 (+ 30us) write_op.cc:435] Acquired schema lock
0808 13:35:05.836492 (+ 16us) tablet.cc:660] Decoding operations
0808 13:35:05.836873 (+ 381us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:05.836916 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:05.836940 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:05.837254 (+ 314us) tablet.cc:699] Row locks acquired
0808 13:35:05.837280 (+ 26us) write_op.cc:260] PREPARE: finished
0808 13:35:05.837409 (+ 129us) write_op.cc:270] Start()
0808 13:35:05.837515 (+ 106us) write_op.cc:276] Timestamp: P: 1786196105837322 usec, L: 0
0808 13:35:05.837533 (+ 18us) op_driver.cc:331] REPLICATION: starting
0808 13:35:05.838157 (+ 624us) log.cc:842] Serialized 724 byte log entry
0808 13:35:06.047616 (+209459us) op_driver.cc:444] REPLICATION: finished
0808 13:35:06.052398 (+ 4782us) write_op.cc:301] APPLY: starting
0808 13:35:06.052452 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:06.052637 (+ 185us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:06.052886 (+ 249us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:06.053252 (+ 366us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:06.053580 (+ 328us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:06.092558 (+ 38978us) write_op.cc:312] APPLY: finished
0808 13:35:06.093067 (+ 509us) log.cc:842] Serialized 28 byte log entry
0808 13:35:06.164185 (+ 71118us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:06.164451 (+ 266us) write_op.cc:454] Released schema lock
0808 13:35:06.164569 (+ 118us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:06.166286 (+ 1717us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:06.166538 (+ 252us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3898,"child_traces":[["op",{"apply.queue_time_us":4616,"apply.run_cpu_time_us":3539,"apply.run_wall_time_us":114579,"num_ops":2,"prepare.queue_time_us":15173,"prepare.run_cpu_time_us":2415,"prepare.run_wall_time_us":2416,"raft.queue_time_us":2188,"raft.run_cpu_time_us":9,"raft.run_wall_time_us":9,"replication_time_us":209944}]]}
W20260808 13:35:06.236533 6125 log.cc:924] Time spent T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: Append to log took a long time: real 0.490s user 0.000s sys 0.002s
I20260808 13:35:06.240528 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aaf4fe334334b5db2fcc1bc8992c113" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:06.245193 6033 tablet_bootstrap.cc:492] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:06.248668 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c140276a5d7438b8c63e02daecd3f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:06.248013 5642 raft_consensus.cc:3037] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:06.273545 6026 tablet_bootstrap.cc:654] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:06.292675 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_17e6c38083384a1b94be67b7a240e85b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:06.294229 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_17e6c38083384a1b94be67b7a240e85b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:06.295181 5642 raft_consensus.cc:2454] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:06.295333 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_414cb3a438f24b8181e2d5d53bd1f3e4" } modify_external_catalogs: true
I20260808 13:35:06.295873 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_414cb3a438f24b8181e2d5d53bd1f3e4" } modify_external_catalogs: true
I20260808 13:35:06.298976 5644 raft_consensus.cc:2454] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:06.299412 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_b23ea067bd8149539074d42c1596a2aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:06.301081 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b23ea067bd8149539074d42c1596a2aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:06.303545 6134 tablet_replica.cc:330] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:06.309677 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_56d20c0c83ee43d7bbad6a06ba148795"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:06.311767 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56d20c0c83ee43d7bbad6a06ba148795 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:06.312300 5772 leader_election.cc:302] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:06.320901 6103 raft_consensus.cc:2781] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:06.321882 6103 raft_consensus.cc:686] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:06.323473 6103 consensus_queue.cc:237] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.343672 6120 raft_consensus.cc:482] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:06.344236 6120 raft_consensus.cc:504] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.346215 6120 leader_election.cc:288] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
W20260808 13:35:06.348028 4840 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ddfd9e3f002445738df24ebb0a333440 has no consensus state
I20260808 13:35:06.359788 6134 raft_consensus.cc:2229] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:06.360298 6134 raft_consensus.cc:2258] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:06.365384 6132 tablet_replica.cc:330] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:06.366987 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59e1f5f36a2480bbb7288a1449a54de" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:06.369011 6132 raft_consensus.cc:2229] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:06.369514 6132 raft_consensus.cc:2258] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:06.372208 6132 ts_tablet_manager.cc:1905] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:06.374387 6033 tablet_bootstrap.cc:654] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:06.375243 5772 leader_election.cc:341] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:06.377018 6134 ts_tablet_manager.cc:1905] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:06.399467 6026 tablet_bootstrap.cc:492] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:06.400084 6026 ts_tablet_manager.cc:1397] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.710s user 0.010s sys 0.011s
I20260808 13:35:06.402969 6026 raft_consensus.cc:348] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.403751 6026 raft_consensus.cc:374] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:06.404079 6026 raft_consensus.cc:729] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:06.404726 6026 consensus_queue.cc:260] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.406777 6026 ts_tablet_manager.cc:1428] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 13:35:06.407648 6026 tablet_bootstrap.cc:492] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:06.413432 5285 leader_election.cc:302] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:06.415117 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59e1f5f36a2480bbb7288a1449a54de" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:06.416774 5771 leader_election.cc:341] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:06.417344 5771 leader_election.cc:302] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:06.418181 5771 raft_consensus.cc:2670] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260808 13:35:06.420765 6144 raft_consensus.cc:482] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:06.421253 6144 raft_consensus.cc:504] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.431566 4832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_488d26ecb7bd4a83a08f3722b9297990" } modify_external_catalogs: true
I20260808 13:35:06.432341 4832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_488d26ecb7bd4a83a08f3722b9297990" } modify_external_catalogs: true
I20260808 13:35:06.437562 6026 tablet_bootstrap.cc:654] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:06.437407 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet c59e1f5f36a2480bbb7288a1449a54de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c59e1f5f36a2480bbb7288a1449a54de already in progress: creating tablet
W20260808 13:35:06.439112 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 1c140276a5d7438b8c63e02daecd3f80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c140276a5d7438b8c63e02daecd3f80 already in progress: creating tablet
I20260808 13:35:06.449605 6114 consensus_queue.cc:1035] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:06.452407 6033 tablet_bootstrap.cc:492] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:06.452883 6033 ts_tablet_manager.cc:1397] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.208s user 0.007s sys 0.003s
I20260808 13:35:06.452832 6134 ts_tablet_manager.cc:1918] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:06.453426 6134 log.cc:1196] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/c59e1f5f36a2480bbb7288a1449a54de
I20260808 13:35:06.455442 6134 ts_tablet_manager.cc:1939] T c59e1f5f36a2480bbb7288a1449a54de P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
W20260808 13:35:06.459208 4823 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c -> Peer 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135): Couldn't send request to peer 87e8b888041b4135af55792050defcb5. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260808 13:35:06.467800 6143 raft_consensus.cc:2781] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
W20260808 13:35:06.468884 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 55509.2us
I20260808 13:35:06.472127 5386 raft_consensus.cc:2454] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:06.475240 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106aef8e805d4a0ebaaf2bdd808bbfab" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:06.474083 6144 leader_election.cc:288] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:06.494969 6143 raft_consensus.cc:686] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:06.499600 6143 consensus_queue.cc:237] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.511466 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 6d8018bf8ff843d8818fbba9473acf47 (DEFAULT_TABLE table=default.Table_de035120cda84e3da95340a3967f4396 [id=fc56814206154b1c9d77c3b160d25aa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.520274 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d8018bf8ff843d8818fbba9473acf47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.532485 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet a1a5a9f860b34797825be7568a0b4ccb (table default.Table_18666f1bccbb4a41ba39aafa9354326b [id=807c9b982b81419e903344d491fabe09]). Expected version 2 got 0
I20260808 13:35:06.514480 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 6d8018bf8ff843d8818fbba9473acf47 (DEFAULT_TABLE table=default.Table_de035120cda84e3da95340a3967f4396 [id=fc56814206154b1c9d77c3b160d25aa8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:06.535135 4835 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1a5a9f860b34797825be7568a0b4ccb consensus state has no leader
I20260808 13:35:06.536569 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d8018bf8ff843d8818fbba9473acf47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.541224 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 556c57c5b9704b46b142796db2506027 (DEFAULT_TABLE table=default.Table_41aa30b0f19f470ea3c1f10a3bf72c73 [id=aa9156d7ba2d452e81c86b72891b6f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.547639 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556c57c5b9704b46b142796db2506027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.553772 4835 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_3009d485300e4bc78022312d9ddd9a34" } new_table_name: "default.Table_6cb245b3d74d4347843035902b8d49d1" modify_external_catalogs: true
W20260808 13:35:06.556262 6046 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.537s user 0.005s sys 0.006s
W20260808 13:35:06.556540 6046 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.538s user 0.005s sys 0.006s
I20260808 13:35:06.559239 6046 tablet_bootstrap.cc:492] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:06.559710 6046 ts_tablet_manager.cc:1397] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.676s user 0.019s sys 0.013s
I20260808 13:35:06.536083 6033 raft_consensus.cc:348] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.572304 6033 raft_consensus.cc:374] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:06.572642 6033 raft_consensus.cc:729] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:06.575562 4838 catalog_manager.cc:5495] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:06.585958 5351 tablet_service.cc:1467] Processing CreateTablet for tablet ddfd9e3f002445738df24ebb0a333440 (DEFAULT_TABLE table=default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.587905 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddfd9e3f002445738df24ebb0a333440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.592471 5345 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aaf4fe334334b5db2fcc1bc8992c113 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:06.573393 6033 consensus_queue.cc:260] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.596962 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 556c57c5b9704b46b142796db2506027 (DEFAULT_TABLE table=default.Table_41aa30b0f19f470ea3c1f10a3bf72c73 [id=aa9156d7ba2d452e81c86b72891b6f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.602540 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aaf4fe334334b5db2fcc1bc8992c113 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:06.602748 6033 ts_tablet_manager.cc:1428] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.150s user 0.007s sys 0.003s
I20260808 13:35:06.605870 6026 tablet_bootstrap.cc:492] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:06.606365 6026 ts_tablet_manager.cc:1397] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.199s user 0.021s sys 0.014s
I20260808 13:35:06.607216 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106aef8e805d4a0ebaaf2bdd808bbfab" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:06.609890 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 6d8018bf8ff843d8818fbba9473acf47 (DEFAULT_TABLE table=default.Table_de035120cda84e3da95340a3967f4396 [id=fc56814206154b1c9d77c3b160d25aa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.611467 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d8018bf8ff843d8818fbba9473acf47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.618857 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556c57c5b9704b46b142796db2506027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.619228 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 556c57c5b9704b46b142796db2506027 (DEFAULT_TABLE table=default.Table_41aa30b0f19f470ea3c1f10a3bf72c73 [id=aa9156d7ba2d452e81c86b72891b6f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.621412 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556c57c5b9704b46b142796db2506027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.623863 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet c59e1f5f36a2480bbb7288a1449a54de (table default.Table_5f0e676e2cf14d31b0439a4fdd39a58f [id=9a99bed19d724319bfaae469dca0ed62]) successfully deleted
I20260808 13:35:06.628298 6026 raft_consensus.cc:348] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.629243 6026 raft_consensus.cc:374] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:06.629575 6026 raft_consensus.cc:729] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:06.630090 5841 tablet_service.cc:1467] Processing CreateTablet for tablet ddfd9e3f002445738df24ebb0a333440 (DEFAULT_TABLE table=default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.632411 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddfd9e3f002445738df24ebb0a333440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.638170 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 60f8863bc32e49399940912444bcb972 (DEFAULT_TABLE table=default.Table_ea98c448b3f3473683c7e7e0ac5ae2be [id=46d67ef595cf4e9d9516c7775cc56ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.640220 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f8863bc32e49399940912444bcb972. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:06.640810 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 1c140276a5d7438b8c63e02daecd3f80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c140276a5d7438b8c63e02daecd3f80 already in progress: creating tablet
I20260808 13:35:06.639415 5613 tablet_service.cc:1467] Processing CreateTablet for tablet ddfd9e3f002445738df24ebb0a333440 (DEFAULT_TABLE table=default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.644172 5615 tablet_service.cc:1467] Processing CreateTablet for tablet 60f8863bc32e49399940912444bcb972 (DEFAULT_TABLE table=default.Table_ea98c448b3f3473683c7e7e0ac5ae2be [id=46d67ef595cf4e9d9516c7775cc56ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.646296 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f8863bc32e49399940912444bcb972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.659473 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddfd9e3f002445738df24ebb0a333440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.659973 6033 tablet_bootstrap.cc:492] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:06.679366 5611 tablet_service.cc:1467] Processing CreateTablet for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (DEFAULT_TABLE table=default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.682096 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2a95712de254c13b3b6dc4fe5cb15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.630363 6026 consensus_queue.cc:260] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.692519 6026 ts_tablet_manager.cc:1428] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.075s user 0.008s sys 0.000s
I20260808 13:35:06.712821 6046 raft_consensus.cc:348] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.713557 6046 raft_consensus.cc:374] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:06.713828 6046 raft_consensus.cc:729] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:06.720329 5386 raft_consensus.cc:3037] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:06.721411 5345 tablet_service.cc:1467] Processing CreateTablet for tablet 60f8863bc32e49399940912444bcb972 (DEFAULT_TABLE table=default.Table_ea98c448b3f3473683c7e7e0ac5ae2be [id=46d67ef595cf4e9d9516c7775cc56ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.714457 6046 consensus_queue.cc:260] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.736837 6033 tablet_bootstrap.cc:654] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:06.741487 6026 tablet_bootstrap.cc:492] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:06.743258 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a1a5a9f860b34797825be7568a0b4ccb (table default.Table_34602dc423cc4b8b8cc125d6c1ce5452 [id=807c9b982b81419e903344d491fabe09], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet a1a5a9f860b34797825be7568a0b4ccb consensus state has no leader
I20260808 13:35:06.747342 6111 raft_consensus.cc:482] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:06.703151 6131 tablet_replica.cc:330] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:06.731174 5285 leader_election.cc:302] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:06.726969 5356 tablet_service.cc:1467] Processing CreateTablet for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (DEFAULT_TABLE table=default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.734220 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f8863bc32e49399940912444bcb972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.748000 6111 raft_consensus.cc:504] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.753582 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8948ce98e7a941f59fa9932fe241638d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:06.756256 5551 leader_election.cc:341] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:06.758373 6111 leader_election.cc:288] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:06.759099 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2a95712de254c13b3b6dc4fe5cb15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.763612 6131 raft_consensus.cc:2229] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 13:35:06.767261 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 56864.7us
I20260808 13:35:06.770900 6131 raft_consensus.cc:2258] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:06.773847 6131 ts_tablet_manager.cc:1905] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:06.783002 6046 ts_tablet_manager.cc:1428] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.223s user 0.006s sys 0.003s
I20260808 13:35:06.800128 6046 tablet_bootstrap.cc:492] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:06.822404 6154 raft_consensus.cc:2726] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:06.832926 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_7ad3b44b6be7476881db975f93533b24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:06.835950 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ad3b44b6be7476881db975f93533b24 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:06.836668 4833 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 6f756ddacbd64194a02d957bb3f3b269
I20260808 13:35:06.841815 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_bd6c3a9f4c304c79aa6fa4854e4df372"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:06.856223 6026 tablet_bootstrap.cc:654] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:06.856652 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd6c3a9f4c304c79aa6fa4854e4df372 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:06.850330 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_98d1673dbc6c43a78c1353fb1b151feb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:06.863222 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98d1673dbc6c43a78c1353fb1b151feb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:06.852083 4839 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_e4d8738a572f475a9becc0e4e90d6c1a" } new_table_name: "default.Table_703d70b501fa4188a210efda24546a02" modify_external_catalogs: true
I20260808 13:35:06.872267 6111 consensus_queue.cc:1035] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:06.918205 6154 raft_consensus.cc:482] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:06.919667 6154 raft_consensus.cc:504] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:06.930908 5285 leader_election.cc:341] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: c59e1f5f36a2480bbb7288a1449a54de
I20260808 13:35:06.932636 5843 tablet_service.cc:1467] Processing CreateTablet for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (DEFAULT_TABLE table=default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:06.941452 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59e1f5f36a2480bbb7288a1449a54de" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:06.943306 5283 leader_election.cc:341] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: Tablet not RUNNING: STOPPED
I20260808 13:35:06.943986 5283 leader_election.cc:302] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:06.945745 6143 raft_consensus.cc:2726] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:06.946815 6154 leader_election.cc:288] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:06.954101 6046 tablet_bootstrap.cc:654] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:06.911307 6103 raft_consensus.cc:482] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:06.942227 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8948ce98e7a941f59fa9932fe241638d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:06.959337 5880 raft_consensus.cc:2454] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:06.961467 5550 leader_election.cc:302] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:06.957827 6103 raft_consensus.cc:504] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.964617 6026 tablet_bootstrap.cc:492] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:06.965123 6026 ts_tablet_manager.cc:1397] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.224s user 0.015s sys 0.004s
I20260808 13:35:06.966694 6103 leader_election.cc:288] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:06.967561 6111 raft_consensus.cc:2781] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:06.968210 4832 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet d68644fb5769402ca7f8fb461c6c7664
I20260808 13:35:06.968137 6111 raft_consensus.cc:482] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:06.968485 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a5a9f860b34797825be7568a0b4ccb" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:06.968717 6111 raft_consensus.cc:3037] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:06.968757 6026 raft_consensus.cc:348] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.934479 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2a95712de254c13b3b6dc4fe5cb15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:06.969718 5642 raft_consensus.cc:2379] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:06.969545 6026 raft_consensus.cc:374] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:06.985371 6026 raft_consensus.cc:729] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:06.986311 6026 consensus_queue.cc:260] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:06.996901 6026 ts_tablet_manager.cc:1428] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.031s user 0.004s sys 0.004s
I20260808 13:35:06.997808 6026 tablet_bootstrap.cc:492] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:07.001539 6111 raft_consensus.cc:504] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.003278 6132 ts_tablet_manager.cc:1918] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:07.003975 6132 log.cc:1196] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/c59e1f5f36a2480bbb7288a1449a54de
I20260808 13:35:07.005105 6132 ts_tablet_manager.cc:1939] T c59e1f5f36a2480bbb7288a1449a54de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:07.037875 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet c59e1f5f36a2480bbb7288a1449a54de (table default.Table_5f0e676e2cf14d31b0439a4fdd39a58f [id=9a99bed19d724319bfaae469dca0ed62]) successfully deleted
W20260808 13:35:07.043066 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 1c140276a5d7438b8c63e02daecd3f80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c140276a5d7438b8c63e02daecd3f80 already in progress: creating tablet
I20260808 13:35:07.043623 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet a1a5a9f860b34797825be7568a0b4ccb (table default.Table_34602dc423cc4b8b8cc125d6c1ce5452 [id=807c9b982b81419e903344d491fabe09]). Expected version 3 got 0
I20260808 13:35:07.046955 6033 tablet_bootstrap.cc:492] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:07.047672 6033 ts_tablet_manager.cc:1397] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.388s user 0.006s sys 0.016s
I20260808 13:35:07.035115 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aaf4fe334334b5db2fcc1bc8992c113 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:04 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:07.038584 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet c59e1f5f36a2480bbb7288a1449a54de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:07.051863 6134 tablet_replica.cc:330] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:07.053081 6134 raft_consensus.cc:2229] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:07.053727 6134 raft_consensus.cc:2258] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:07.062041 6134 ts_tablet_manager.cc:1905] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:07.073271 6033 raft_consensus.cc:348] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.074216 6033 raft_consensus.cc:374] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:07.062098 4833 catalog_manager.cc:5495] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:07.081178 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet a1a5a9f860b34797825be7568a0b4ccb (table default.Table_34602dc423cc4b8b8cc125d6c1ce5452 [id=807c9b982b81419e903344d491fabe09]). Expected version 3 got 0
I20260808 13:35:07.086208 6111 leader_election.cc:288] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:07.078945 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet c59e1f5f36a2480bbb7288a1449a54de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:07.079473 6132 tablet_replica.cc:330] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:07.081530 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8948ce98e7a941f59fa9932fe241638d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:07.087352 5880 raft_consensus.cc:3037] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:07.089066 4838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_34602dc423cc4b8b8cc125d6c1ce5452" } modify_external_catalogs: true
I20260808 13:35:07.089727 4838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_34602dc423cc4b8b8cc125d6c1ce5452" } modify_external_catalogs: true
I20260808 13:35:06.970430 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a5a9f860b34797825be7568a0b4ccb" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:07.059398 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8948ce98e7a941f59fa9932fe241638d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:07.099913 5880 raft_consensus.cc:2454] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:07.050791 6026 tablet_bootstrap.cc:654] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:07.102643 5771 leader_election.cc:302] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:07.103974 6103 raft_consensus.cc:3037] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:07.091969 6052 raft_consensus.cc:482] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:07.105047 6052 raft_consensus.cc:504] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.108227 6052 leader_election.cc:288] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:07.100468 5551 leader_election.cc:341] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:07.114464 5550 leader_election.cc:302] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:07.122071 6052 raft_consensus.cc:2781] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:07.122902 6052 raft_consensus.cc:686] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:07.101702 6132 raft_consensus.cc:2229] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:07.124480 6132 pending_rounds.cc:62] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Trying to abort 1 pending ops.
I20260808 13:35:07.124661 6052 consensus_queue.cc:237] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.127391 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edfa66acceea4b54911d78ee16609ef7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:07.128563 6132 pending_rounds.cc:68] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259251514732544 op_type: NO_OP noop_request { }
W20260808 13:35:07.129453 5551 leader_election.cc:341] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:07.133555 6111 raft_consensus.cc:482] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:07.134305 6132 raft_consensus.cc:2866] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:35:07.134263 6111 raft_consensus.cc:504] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.135895 6132 raft_consensus.cc:2258] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:07.138181 4835 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 78d492406c8a4c4186131e02a1559cb4 has no consensus state
I20260808 13:35:07.142488 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e36f7753d9d4c43afc3c9b253a03846" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:07.144483 5551 leader_election.cc:341] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:07.102164 6033 raft_consensus.cc:729] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:07.148409 6033 consensus_queue.cc:260] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.125733 6103 raft_consensus.cc:2726] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:07.139401 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edfa66acceea4b54911d78ee16609ef7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:07.161799 5550 leader_election.cc:341] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:07.163164 6033 ts_tablet_manager.cc:1428] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.115s user 0.008s sys 0.000s
I20260808 13:35:07.164832 6033 tablet_bootstrap.cc:492] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:07.170730 5550 leader_election.cc:302] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:07.172672 6111 leader_election.cc:288] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:07.176683 6132 ts_tablet_manager.cc:1905] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:07.201045 4823 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:37135, user_credentials={real_user=slave}} blocked reactor thread for 98035.4us
I20260808 13:35:07.207736 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_41aa30b0f19f470ea3c1f10a3bf72c73" } modify_external_catalogs: true
I20260808 13:35:07.208779 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_41aa30b0f19f470ea3c1f10a3bf72c73" } modify_external_catalogs: true
I20260808 13:35:07.212443 6185 raft_consensus.cc:2726] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:07.214821 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_577782e78de947ea87dd0e0bf0643c25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:07.215258 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 53521.2us
W20260808 13:35:07.216437 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_577782e78de947ea87dd0e0bf0643c25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:07.217064 6026 tablet_bootstrap.cc:492] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:07.217535 6026 ts_tablet_manager.cc:1397] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.220s user 0.020s sys 0.005s
I20260808 13:35:07.217921 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e36f7753d9d4c43afc3c9b253a03846" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:07.218851 5882 raft_consensus.cc:2454] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:07.220757 5550 leader_election.cc:302] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:07.221534 6026 raft_consensus.cc:348] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:07.225348 6026 raft_consensus.cc:374] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:07.226073 6026 raft_consensus.cc:729] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:07.229764 6033 tablet_bootstrap.cc:654] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:07.223596 6180 raft_consensus.cc:2781] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:07.233225 6180 raft_consensus.cc:482] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:07.233728 6180 raft_consensus.cc:3037] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:07.232005 6026 consensus_queue.cc:260] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:07.246215 6026 ts_tablet_manager.cc:1428] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.028s user 0.004s sys 0.003s
I20260808 13:35:07.247593 6026 tablet_bootstrap.cc:492] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:07.309967 6046 tablet_bootstrap.cc:492] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:07.311775 6052 consensus_queue.cc:1035] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:07.335778 6134 ts_tablet_manager.cc:1918] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:07.336603 6134 log.cc:1196] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/5aaf4fe334334b5db2fcc1bc8992c113
I20260808 13:35:07.337831 6134 ts_tablet_manager.cc:1939] T 5aaf4fe334334b5db2fcc1bc8992c113 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:07.341831 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 5aaf4fe334334b5db2fcc1bc8992c113 (table default.Table_c08ec3a916d44f25b61fa57b88efcbcc [id=3d42b09e05b3458caffce3214d58f93a]) successfully deleted
I20260808 13:35:07.342199 6026 tablet_bootstrap.cc:654] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:07.344779 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_de035120cda84e3da95340a3967f4396" } new_table_name: "default.Table_810abec89b21407191098eb04ea650e2" modify_external_catalogs: true
I20260808 13:35:07.355403 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_04ef257f28bf43a3b2fec07582b495ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:07.358122 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04ef257f28bf43a3b2fec07582b495ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:07.310544 6046 ts_tablet_manager.cc:1397] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.527s user 0.011s sys 0.006s
I20260808 13:35:07.360333 6180 raft_consensus.cc:504] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.373406 6033 tablet_bootstrap.cc:492] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:07.373831 6033 ts_tablet_manager.cc:1397] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.209s user 0.022s sys 0.019s
I20260808 13:35:07.374260 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e36f7753d9d4c43afc3c9b253a03846" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:07.375233 5882 raft_consensus.cc:3037] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:07.390389 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet c59e1f5f36a2480bbb7288a1449a54de because the tablet was not found. No further retry: Not found: Tablet not found: c59e1f5f36a2480bbb7288a1449a54de
I20260808 13:35:07.388900 6180 leader_election.cc:288] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:07.393553 6033 raft_consensus.cc:348] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.394685 6046 raft_consensus.cc:348] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.395754 6046 raft_consensus.cc:374] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:07.396042 6046 raft_consensus.cc:729] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:07.396677 6046 consensus_queue.cc:260] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.399809 6033 raft_consensus.cc:374] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:07.400259 6033 raft_consensus.cc:729] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:07.399710 5882 raft_consensus.cc:2454] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:07.399255 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e36f7753d9d4c43afc3c9b253a03846" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:07.403049 5385 raft_consensus.cc:3037] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:07.403018 6033 consensus_queue.cc:260] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.404975 5550 leader_election.cc:302] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:07.406852 6180 raft_consensus.cc:2781] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:07.407680 6180 raft_consensus.cc:686] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:07.411649 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a5dcf9a6d69a4c1c94e6c32081b818b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:07.413673 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5dcf9a6d69a4c1c94e6c32081b818b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:07.417557 6131 ts_tablet_manager.cc:1918] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:07.418035 6131 log.cc:1196] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/5aaf4fe334334b5db2fcc1bc8992c113
I20260808 13:35:07.413230 6180 consensus_queue.cc:237] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.422233 6046 ts_tablet_manager.cc:1428] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.062s user 0.008s sys 0.000s
I20260808 13:35:07.428457 6033 ts_tablet_manager.cc:1428] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.054s user 0.003s sys 0.007s
I20260808 13:35:07.436141 6033 tablet_bootstrap.cc:492] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:07.436168 6132 ts_tablet_manager.cc:1918] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:07.444094 6132 log.cc:1196] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/5aaf4fe334334b5db2fcc1bc8992c113
I20260808 13:35:07.448943 4837 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5eabc9a0e2b74737ac3e20af0fa63336 [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 1 got 0
I20260808 13:35:07.451574 6132 ts_tablet_manager.cc:1939] T 5aaf4fe334334b5db2fcc1bc8992c113 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:07.455101 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 5aaf4fe334334b5db2fcc1bc8992c113 (table default.Table_c08ec3a916d44f25b61fa57b88efcbcc [id=3d42b09e05b3458caffce3214d58f93a]) successfully deleted
W20260808 13:35:07.460772 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet c59e1f5f36a2480bbb7288a1449a54de because the tablet was not found. No further retry: Not found: Tablet not found: c59e1f5f36a2480bbb7288a1449a54de
I20260808 13:35:07.464360 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5f83ffeafb6340ec8f90c0eb040ec349"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:07.466367 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f83ffeafb6340ec8f90c0eb040ec349 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:07.454463 6033 tablet_bootstrap.cc:654] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:07.483431 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5eabc9a0e2b74737ac3e20af0fa63336 [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 1 got 0
I20260808 13:35:07.508369 6033 tablet_bootstrap.cc:492] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:07.508888 6033 ts_tablet_manager.cc:1397] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.073s user 0.008s sys 0.003s
I20260808 13:35:07.508729 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 328d361c336848689c6906ca6fe2ccc3 (DEFAULT_TABLE table=default.Table_b23ea067bd8149539074d42c1596a2aa [id=3a77dc355690462ba633f54fadbeb544]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.509670 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 328d361c336848689c6906ca6fe2ccc3 (DEFAULT_TABLE table=default.Table_b23ea067bd8149539074d42c1596a2aa [id=3a77dc355690462ba633f54fadbeb544]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.510502 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 328d361c336848689c6906ca6fe2ccc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:07.511922 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 328d361c336848689c6906ca6fe2ccc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:07.511693 6033 raft_consensus.cc:348] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:07.523427 6033 raft_consensus.cc:374] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:07.523833 6033 raft_consensus.cc:729] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:07.488638 4832 catalog_manager.cc:5495] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:07.524603 6033 consensus_queue.cc:260] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:07.525501 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5f18e5b0a4c445a88b7717441edb323f (table default.Table_e79e02d050604203b805744df0be2c8c [id=3540defcda954120ba76fe9e4c3869a8]). Expected version 3 got 0
I20260808 13:35:07.526522 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]). Expected version 1 got 0
I20260808 13:35:07.534168 6033 ts_tablet_manager.cc:1428] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.025s user 0.007s sys 0.001s
I20260808 13:35:07.539496 6046 tablet_bootstrap.cc:492] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:07.544618 5845 tablet_service.cc:1467] Processing CreateTablet for tablet e7616d81aa924933a9b24d44004725fa (DEFAULT_TABLE table=default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.545631 5611 tablet_service.cc:1467] Processing CreateTablet for tablet e7616d81aa924933a9b24d44004725fa (DEFAULT_TABLE table=default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.546479 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7616d81aa924933a9b24d44004725fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:07.547716 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7616d81aa924933a9b24d44004725fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:07.559541 6033 tablet_bootstrap.cc:492] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:07.562759 6175 raft_consensus.cc:482] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:07.563422 6175 raft_consensus.cc:504] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:07.567766 6175 leader_election.cc:288] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:07.569725 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75740d7986e0439a8677dc14d29ad0e5" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:07.570776 5642 raft_consensus.cc:2454] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:07.571467 6052 consensus_queue.cc:1035] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:07.576633 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75740d7986e0439a8677dc14d29ad0e5" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:07.579226 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 328d361c336848689c6906ca6fe2ccc3 (DEFAULT_TABLE table=default.Table_b23ea067bd8149539074d42c1596a2aa [id=3a77dc355690462ba633f54fadbeb544]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.582412 5772 leader_election.cc:302] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:07.585839 6175 raft_consensus.cc:2781] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:07.587779 6175 raft_consensus.cc:482] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:07.588506 6175 raft_consensus.cc:3037] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:07.596019 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 78d492406c8a4c4186131e02a1559cb4 (DEFAULT_TABLE table=default.Table_6cb245b3d74d4347843035902b8d49d1 [id=c0806126263c41b68137cad5e21de125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.597795 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d492406c8a4c4186131e02a1559cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:07.597574 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 78d492406c8a4c4186131e02a1559cb4 (DEFAULT_TABLE table=default.Table_6cb245b3d74d4347843035902b8d49d1 [id=c0806126263c41b68137cad5e21de125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.599380 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d492406c8a4c4186131e02a1559cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:07.620592 6026 tablet_bootstrap.cc:492] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:07.621124 6026 ts_tablet_manager.cc:1397] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.374s user 0.011s sys 0.027s
W20260808 13:35:07.621261 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6d8018bf8ff843d8818fbba9473acf47 consensus state has no leader
I20260808 13:35:07.628149 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 556c57c5b9704b46b142796db2506027
I20260808 13:35:07.629868 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_203fa9f53a2345519f4a5825d2661b97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:07.632895 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_203fa9f53a2345519f4a5825d2661b97 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:07.633728 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 556c57c5b9704b46b142796db2506027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:07.632669 6026 raft_consensus.cc:348] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:07.636723 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 556c57c5b9704b46b142796db2506027 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 556c57c5b9704b46b142796db2506027 already in progress: creating tablet
I20260808 13:35:07.639647 5615 tablet_service.cc:1514] Processing DeleteTablet for tablet 556c57c5b9704b46b142796db2506027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:07.641913 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 556c57c5b9704b46b142796db2506027 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 556c57c5b9704b46b142796db2506027 already in progress: creating tablet
I20260808 13:35:07.643352 6026 raft_consensus.cc:374] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:07.643671 6026 raft_consensus.cc:729] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:07.644632 6026 consensus_queue.cc:260] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:07.679371 5345 tablet_service.cc:1467] Processing CreateTablet for tablet e7616d81aa924933a9b24d44004725fa (DEFAULT_TABLE table=default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.680698 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7616d81aa924933a9b24d44004725fa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:07.680369 5771 leader_election.cc:341] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:07.680452 6175 raft_consensus.cc:504] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:07.685671 6033 tablet_bootstrap.cc:654] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:07.667321 6131 ts_tablet_manager.cc:1939] T 5aaf4fe334334b5db2fcc1bc8992c113 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:07.695065 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 328d361c336848689c6906ca6fe2ccc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:07.700886 6026 ts_tablet_manager.cc:1428] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.075s user 0.002s sys 0.006s
I20260808 13:35:07.701975 6026 tablet_bootstrap.cc:492] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:07.705229 5385 raft_consensus.cc:2454] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:07.709637 6131 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 25) took 1117 ms. Trace:
I20260808 13:35:07.712982 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 556c57c5b9704b46b142796db2506027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:07.713472 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 78d492406c8a4c4186131e02a1559cb4 (DEFAULT_TABLE table=default.Table_6cb245b3d74d4347843035902b8d49d1 [id=c0806126263c41b68137cad5e21de125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.715492 5355 tablet_service.cc:1467] Processing CreateTablet for tablet f26e054d1992452396d3f7334ba3602c (DEFAULT_TABLE table=default.Table_17e6c38083384a1b94be67b7a240e85b [id=b301b3d4964b41009ed2903dcbefdfae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.717060 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d492406c8a4c4186131e02a1559cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:07.710187 6131 rpcz_store.cc:276] 0808 13:35:06.592097 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:06.592273 (+ 176us) service_pool.cc:227] Handling call
0808 13:35:06.593281 (+ 1008us) ts_tablet_manager.cc:1145] Deleting tablet 5aaf4fe334334b5db2fcc1bc8992c113
0808 13:35:06.593340 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:07.225642 (+632302us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:07.402301 (+176659us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:07.709548 (+307247us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20273,"tablet-delete.queue_time_us":185}
I20260808 13:35:07.719506 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f26e054d1992452396d3f7334ba3602c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:07.723706 6185 raft_consensus.cc:482] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:07.726400 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet a1a5a9f860b34797825be7568a0b4ccb
W20260808 13:35:07.732399 4833 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 26) took 884 ms (client timeout 1000 ms). Trace:
I20260808 13:35:07.735502 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 5aaf4fe334334b5db2fcc1bc8992c113 (table default.Table_c08ec3a916d44f25b61fa57b88efcbcc [id=3d42b09e05b3458caffce3214d58f93a]) successfully deleted
W20260808 13:35:07.736232 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 556c57c5b9704b46b142796db2506027 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 556c57c5b9704b46b142796db2506027 already in progress: creating tablet
I20260808 13:35:07.738916 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a5a9f860b34797825be7568a0b4ccb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:07.733019 4833 rpcz_store.cc:269] 0808 13:35:06.848642 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:06.857803 (+ 9161us) service_pool.cc:227] Handling call
0808 13:35:07.732314 (+874511us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:07.100368 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:07.100721 (+ 353us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:07.100751 (+ 30us) write_op.cc:435] Acquired schema lock
0808 13:35:07.100763 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:07.101008 (+ 245us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:07.101043 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:07.101064 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:07.101253 (+ 189us) tablet.cc:699] Row locks acquired
0808 13:35:07.101268 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:07.101377 (+ 109us) write_op.cc:270] Start()
0808 13:35:07.101496 (+ 119us) write_op.cc:276] Timestamp: P: 1786196107101338 usec, L: 0
0808 13:35:07.101524 (+ 28us) op_driver.cc:331] REPLICATION: starting
0808 13:35:07.105463 (+ 3939us) log.cc:842] Serialized 526 byte log entry
0808 13:35:07.243434 (+137971us) op_driver.cc:444] REPLICATION: finished
0808 13:35:07.274484 (+ 31050us) write_op.cc:301] APPLY: starting
0808 13:35:07.274575 (+ 91us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:07.275258 (+ 683us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:07.275483 (+ 225us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:07.275781 (+ 298us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:07.276042 (+ 261us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:07.276138 (+ 96us) write_op.cc:312] APPLY: finished
0808 13:35:07.276553 (+ 415us) log.cc:842] Serialized 22 byte log entry
0808 13:35:07.276820 (+ 267us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:07.276918 (+ 98us) write_op.cc:454] Released schema lock
0808 13:35:07.276955 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:07.352233 (+ 75278us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b5400021300. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c5c4361 00007f227c695254 00007f227c68e86b 00007f227c68dfd1 00007f227c691dd1 00007f227c691d89 00007f227c691d19 00007f227c691ce1 00007f227c690fdd 0000000000580784 00000000005805b9
0808 13:35:07.352338 (+ 105us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:07.352378 (+ 40us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":30736,"apply.run_cpu_time_us":6057,"apply.run_wall_time_us":78195,"num_ops":1,"prepare.queue_time_us":16842,"prepare.run_cpu_time_us":1980,"prepare.run_wall_time_us":5303,"replication_time_us":141768,"spinlock_wait_cycles":64556800}]]}
I20260808 13:35:07.745580 6131 tablet_replica.cc:330] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:07.746977 6131 raft_consensus.cc:2229] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:07.747994 6131 pending_rounds.cc:62] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: Trying to abort 1 pending ops.
I20260808 13:35:07.754134 6175 leader_election.cc:288] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:07.755144 6200 log.cc:924] Time spent T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.063s user 0.000s sys 0.001s
I20260808 13:35:07.755517 6175 raft_consensus.cc:482] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:07.756124 6175 raft_consensus.cc:504] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.762985 5845 tablet_service.cc:1467] Processing CreateTablet for tablet f26e054d1992452396d3f7334ba3602c (DEFAULT_TABLE table=default.Table_17e6c38083384a1b94be67b7a240e85b [id=b301b3d4964b41009ed2903dcbefdfae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:07.766110 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75740d7986e0439a8677dc14d29ad0e5" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:07.748339 6131 pending_rounds.cc:68] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259252311146496 op_type: NO_OP noop_request { }
I20260808 13:35:07.771698 6131 raft_consensus.cc:2866] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:35:07.772250 6131 raft_consensus.cc:2258] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:07.776929 6046 tablet_bootstrap.cc:654] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:07.784945 4837 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]). Expected version 1 got 0
W20260808 13:35:07.786824 4839 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3e36f7753d9d4c43afc3c9b253a03846 consensus state has no leader
I20260808 13:35:07.787986 4839 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 57) took 1047 ms. Trace:
I20260808 13:35:07.791610 6131 ts_tablet_manager.cc:1905] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:07.795397 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6d8018bf8ff843d8818fbba9473acf47 consensus state has no leader
I20260808 13:35:07.830672 6046 tablet_bootstrap.cc:492] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:07.831231 6046 ts_tablet_manager.cc:1397] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.292s user 0.018s sys 0.006s
I20260808 13:35:07.835312 6046 raft_consensus.cc:348] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:07.836225 6046 raft_consensus.cc:374] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:07.836539 6046 raft_consensus.cc:729] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:07.837335 6046 consensus_queue.cc:260] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:07.843436 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 19) took 759 ms (client timeout 1000 ms). Trace:
I20260808 13:35:07.748469 6185 raft_consensus.cc:504] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:07.843348 4836 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_bd6c3a9f4c304c79aa6fa4854e4df372" } new_table_name: "default.Table_1f0431ee90fa4e2193747a84078f912c" modify_external_catalogs: true
I20260808 13:35:07.859398 6046 ts_tablet_manager.cc:1428] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.028s user 0.009s sys 0.000s
I20260808 13:35:07.860505 6046 tablet_bootstrap.cc:492] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:07.864869 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f26e054d1992452396d3f7334ba3602c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:07.788757 4839 rpcz_store.cc:276] 0808 13:35:06.740454 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:06.740604 (+ 150us) service_pool.cc:227] Handling call
0808 13:35:06.851918 (+111314us) spinlock_profiling.cc:243] Waited 94.1 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff7d6 00007f227bd45634 00007f227bd455c2 00007f227bd454f1 00007f227bd4546c 00007f227bd44722 00007f2279fde94c 00007f2279fddd86
0808 13:35:06.962840 (+110922us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:06.963862 (+ 1022us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:06.965223 (+ 1361us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:06.965587 (+ 364us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:07.213759 (+248172us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:07.787852 (+574093us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:07.020973 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:07.021286 (+ 313us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:07.021306 (+ 20us) write_op.cc:435] Acquired schema lock
0808 13:35:07.021319 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:07.021535 (+ 216us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:07.021566 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:07.021586 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:07.021716 (+ 130us) tablet.cc:699] Row locks acquired
0808 13:35:07.021728 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:07.021827 (+ 99us) write_op.cc:270] Start()
0808 13:35:07.021895 (+ 68us) write_op.cc:276] Timestamp: P: 1786196107021701 usec, L: 0
0808 13:35:07.021910 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:07.022342 (+ 432us) log.cc:842] Serialized 404 byte log entry
0808 13:35:07.206431 (+184089us) op_driver.cc:444] REPLICATION: finished
0808 13:35:07.211191 (+ 4760us) write_op.cc:301] APPLY: starting
0808 13:35:07.211266 (+ 75us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:07.211383 (+ 117us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:07.211558 (+ 175us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:07.211868 (+ 310us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:07.212124 (+ 256us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:07.212223 (+ 99us) write_op.cc:312] APPLY: finished
0808 13:35:07.212622 (+ 399us) log.cc:842] Serialized 22 byte log entry
0808 13:35:07.212907 (+ 285us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:07.213030 (+ 123us) write_op.cc:454] Released schema lock
0808 13:35:07.213086 (+ 56us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:07.213413 (+ 327us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:07.213463 (+ 50us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":216466816,"child_traces":[["op",{"apply.queue_time_us":4551,"apply.run_cpu_time_us":2386,"apply.run_wall_time_us":2474,"num_ops":1,"prepare.queue_time_us":25378,"prepare.run_cpu_time_us":1530,"prepare.run_wall_time_us":1529,"replication_time_us":184446}]]}
W20260808 13:35:07.868263 5771 leader_election.cc:341] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:07.870407 6175 leader_election.cc:288] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:07.882077 6046 tablet_bootstrap.cc:654] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:07.901525 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_1680269ff16246c382d6bccf44f8d52e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:07.902546 6026 tablet_bootstrap.cc:654] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:07.903644 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1680269ff16246c382d6bccf44f8d52e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:07.847106 4832 rpcz_store.cc:269] 0808 13:35:07.083580 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:07.083767 (+ 187us) service_pool.cc:227] Handling call
0808 13:35:07.092462 (+ 8695us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:07.843331 (+750869us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:07.693093 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:07.694880 (+ 1787us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:07.694906 (+ 26us) write_op.cc:435] Acquired schema lock
0808 13:35:07.694918 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:07.695268 (+ 350us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:07.695305 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:07.695325 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:07.695570 (+ 245us) tablet.cc:699] Row locks acquired
0808 13:35:07.695585 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:07.695700 (+ 115us) write_op.cc:270] Start()
0808 13:35:07.695801 (+ 101us) write_op.cc:276] Timestamp: P: 1786196107695652 usec, L: 0
0808 13:35:07.695858 (+ 57us) op_driver.cc:331] REPLICATION: starting
0808 13:35:07.696567 (+ 709us) log.cc:842] Serialized 526 byte log entry
0808 13:35:07.773236 (+ 76669us) op_driver.cc:444] REPLICATION: finished
0808 13:35:07.776752 (+ 3516us) write_op.cc:301] APPLY: starting
0808 13:35:07.776813 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:07.776909 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:07.777021 (+ 112us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:07.777171 (+ 150us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:07.777328 (+ 157us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:07.777372 (+ 44us) write_op.cc:312] APPLY: finished
0808 13:35:07.777555 (+ 183us) log.cc:842] Serialized 22 byte log entry
0808 13:35:07.777743 (+ 188us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:07.777838 (+ 95us) write_op.cc:454] Released schema lock
0808 13:35:07.777880 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:07.834203 (+ 56323us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:07.834275 (+ 72us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":23980,"reported_tablets":5,"spinlock_wait_cycles":13073792,"child_traces":[["op",{"apply.queue_time_us":3376,"apply.run_cpu_time_us":1786,"apply.run_wall_time_us":57758,"num_ops":1,"prepare.queue_time_us":163642,"prepare.run_cpu_time_us":2352,"prepare.run_wall_time_us":3734,"replication_time_us":77276}]]}
I20260808 13:35:07.912674 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_062151f3d1444efc8a42ba6cd06e19c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:07.922192 4839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5f83ffeafb6340ec8f90c0eb040ec349" } modify_external_catalogs: true
I20260808 13:35:07.935832 4839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5f83ffeafb6340ec8f90c0eb040ec349" } modify_external_catalogs: true
I20260808 13:35:07.940928 4832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_e79e02d050604203b805744df0be2c8c" } modify_external_catalogs: true
I20260808 13:35:07.942088 4832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_e79e02d050604203b805744df0be2c8c" } modify_external_catalogs: true
W20260808 13:35:07.945756 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_062151f3d1444efc8a42ba6cd06e19c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:08.008160 4836 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2fa77744bcfb4a5cace730fd9f9fd1ca has no consensus state
I20260808 13:35:08.026110 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c5103bab29d549298c0b01fc0a045e7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:08.029666 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c5103bab29d549298c0b01fc0a045e7a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:08.011008 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 287129us
I20260808 13:35:08.039038 5615 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a5a9f860b34797825be7568a0b4ccb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:08.058902 6185 leader_election.cc:288] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:08.042220 5611 tablet_service.cc:1467] Processing CreateTablet for tablet f26e054d1992452396d3f7334ba3602c (DEFAULT_TABLE table=default.Table_17e6c38083384a1b94be67b7a240e85b [id=b301b3d4964b41009ed2903dcbefdfae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.075780 6134 tablet_replica.cc:330] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:08.077353 6134 raft_consensus.cc:2229] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:08.078292 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f26e054d1992452396d3f7334ba3602c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.082651 5615 tablet_service.cc:1467] Processing CreateTablet for tablet 66d985de8de040829569d1f5783c6fef (DEFAULT_TABLE table=default.Table_577782e78de947ea87dd0e0bf0643c25 [id=d5b4528cac46486e857db6176b4b3a2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.087302 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 345d13cbde6844079c2d1eb80732bb1f (DEFAULT_TABLE table=default.Table_7ad3b44b6be7476881db975f93533b24 [id=cc59d440766d4d2c983f64bca86e7d21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.091904 5616 tablet_service.cc:1467] Processing CreateTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 (DEFAULT_TABLE table=default.Table_a5dcf9a6d69a4c1c94e6c32081b818b7 [id=73f986b821c348e5b9e6cd5a1896eb83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.102686 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c140276a5d7438b8c63e02daecd3f80" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:08.125519 5550 leader_election.cc:341] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:08.135718 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a5a9f860b34797825be7568a0b4ccb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:08.143850 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_509712ada6da421f9e935ee85e481274"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:08.145354 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_509712ada6da421f9e935ee85e481274 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:08.150389 6134 raft_consensus.cc:2258] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:08.153136 5844 tablet_service.cc:1467] Processing CreateTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 (DEFAULT_TABLE table=default.Table_a5dcf9a6d69a4c1c94e6c32081b818b7 [id=73f986b821c348e5b9e6cd5a1896eb83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.155710 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfc9a6ec9a86431fb46e50fcaa658d33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.167208 6134 ts_tablet_manager.cc:1905] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:08.171211 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 345d13cbde6844079c2d1eb80732bb1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.185781 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 66d985de8de040829569d1f5783c6fef (DEFAULT_TABLE table=default.Table_577782e78de947ea87dd0e0bf0643c25 [id=d5b4528cac46486e857db6176b4b3a2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.186586 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 345d13cbde6844079c2d1eb80732bb1f (DEFAULT_TABLE table=default.Table_7ad3b44b6be7476881db975f93533b24 [id=cc59d440766d4d2c983f64bca86e7d21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.187752 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d985de8de040829569d1f5783c6fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.190452 5840 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca (DEFAULT_TABLE table=default.Table_1f0431ee90fa4e2193747a84078f912c [id=c16baabb989244459aec3f7506aaa959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.191674 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 5d388589d386408bb10ca030b87af787 (DEFAULT_TABLE table=default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.192389 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.196062 5836 tablet_service.cc:1467] Processing CreateTablet for tablet bcc0c6ab755c4a6887dabeeef9364201 (DEFAULT_TABLE table=default.Table_56d20c0c83ee43d7bbad6a06ba148795 [id=16cef8437603476b97a8562edba58a41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.197727 4839 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet b3c22f3987b644eebfd872fd5baad280
I20260808 13:35:08.198645 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcc0c6ab755c4a6887dabeeef9364201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.208376 6046 tablet_bootstrap.cc:492] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:08.208318 4832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 5f18e5b0a4c445a88b7717441edb323f
I20260808 13:35:08.208761 6046 ts_tablet_manager.cc:1397] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.348s user 0.036s sys 0.039s
I20260808 13:35:08.218858 6046 raft_consensus.cc:348] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:08.220202 6046 raft_consensus.cc:374] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:08.220739 6046 raft_consensus.cc:729] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:08.221675 6046 consensus_queue.cc:260] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:08.225303 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d388589d386408bb10ca030b87af787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.229869 6046 ts_tablet_manager.cc:1428] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.021s user 0.008s sys 0.000s
I20260808 13:35:08.230808 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f18e5b0a4c445a88b7717441edb323f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:08.232625 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c140276a5d7438b8c63e02daecd3f80" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:08.233852 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 345d13cbde6844079c2d1eb80732bb1f (DEFAULT_TABLE table=default.Table_7ad3b44b6be7476881db975f93533b24 [id=cc59d440766d4d2c983f64bca86e7d21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.234156 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca (DEFAULT_TABLE table=default.Table_1f0431ee90fa4e2193747a84078f912c [id=c16baabb989244459aec3f7506aaa959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.235461 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 345d13cbde6844079c2d1eb80732bb1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.236271 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.243746 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 5d388589d386408bb10ca030b87af787 (DEFAULT_TABLE table=default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.245447 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d388589d386408bb10ca030b87af787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.276590 4839 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c64b1240bf04403d83f297ac2d40590c" } new_table_name: "default.Table_6db98efa64744617a1464e55349e0f0f" modify_external_catalogs: true
I20260808 13:35:08.284715 5345 tablet_service.cc:1467] Processing CreateTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 (DEFAULT_TABLE table=default.Table_a5dcf9a6d69a4c1c94e6c32081b818b7 [id=73f986b821c348e5b9e6cd5a1896eb83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.287564 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 66d985de8de040829569d1f5783c6fef (DEFAULT_TABLE table=default.Table_577782e78de947ea87dd0e0bf0643c25 [id=d5b4528cac46486e857db6176b4b3a2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.296375 5346 tablet_service.cc:1467] Processing CreateTablet for tablet bcc0c6ab755c4a6887dabeeef9364201 (DEFAULT_TABLE table=default.Table_56d20c0c83ee43d7bbad6a06ba148795 [id=16cef8437603476b97a8562edba58a41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.298209 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_114cb957447a4943806bdadc6ca86549"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:08.302321 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_114cb957447a4943806bdadc6ca86549 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:08.300209 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 (DEFAULT_TABLE table=default.Table_04ef257f28bf43a3b2fec07582b495ac [id=3d8a7488d67c47e291b817ee36e99ac0]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:08.203493 5880 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.524s user 0.067s sys 0.132s
W20260808 13:35:08.312297 5772 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.611s user 0.000s sys 0.044s
I20260808 13:35:08.329614 6046 tablet_bootstrap.cc:492] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:08.316839 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d985de8de040829569d1f5783c6fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.339957 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfc9a6ec9a86431fb46e50fcaa658d33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.329437 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfc9a6ec9a86431fb46e50fcaa658d33. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:08.323082 5880 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.644s user 0.067s sys 0.133s
I20260808 13:35:08.311712 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 (DEFAULT_TABLE table=default.Table_04ef257f28bf43a3b2fec07582b495ac [id=3d8a7488d67c47e291b817ee36e99ac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.352051 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca (DEFAULT_TABLE table=default.Table_1f0431ee90fa4e2193747a84078f912c [id=c16baabb989244459aec3f7506aaa959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.353257 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b6e6ce417cc45fd90f5858c30d37a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.353489 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.359055 5346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcc0c6ab755c4a6887dabeeef9364201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.360827 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b6e6ce417cc45fd90f5858c30d37a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.362387 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 345d13cbde6844079c2d1eb80732bb1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.328346 6180 consensus_queue.cc:1035] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260808 13:35:08.366973 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d985de8de040829569d1f5783c6fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.304136 4832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_577782e78de947ea87dd0e0bf0643c25" } new_table_name: "default.Table_85866a1ec57a4727ae173400b18887d6" modify_external_catalogs: true
I20260808 13:35:08.313037 4836 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 3e36f7753d9d4c43afc3c9b253a03846 (table default.Table_703d70b501fa4188a210efda24546a02 [id=e794d9029f6446a996ff766936695c20]). Expected version 1 got 0
I20260808 13:35:08.378109 5609 tablet_service.cc:1467] Processing CreateTablet for tablet 5d388589d386408bb10ca030b87af787 (DEFAULT_TABLE table=default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.378930 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f18e5b0a4c445a88b7717441edb323f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:08.379331 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d388589d386408bb10ca030b87af787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.384320 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edfa66acceea4b54911d78ee16609ef7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:08.384975 5384 raft_consensus.cc:2454] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:08.388963 5384 raft_consensus.cc:3037] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:08.390655 5606 tablet_service.cc:1467] Processing CreateTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 (DEFAULT_TABLE table=default.Table_04ef257f28bf43a3b2fec07582b495ac [id=3d8a7488d67c47e291b817ee36e99ac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.391850 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b6e6ce417cc45fd90f5858c30d37a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.392508 5607 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f18e5b0a4c445a88b7717441edb323f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:08.402165 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1f0431ee90fa4e2193747a84078f912c" } modify_external_catalogs: true
I20260808 13:35:08.402952 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1f0431ee90fa4e2193747a84078f912c" } modify_external_catalogs: true
I20260808 13:35:08.414183 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_34821e686c9b4e988ffdbb94388b2280"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:08.416040 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34821e686c9b4e988ffdbb94388b2280 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:08.416375 6033 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.515s user 0.021s sys 0.028s
W20260808 13:35:08.416656 6033 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.515s user 0.021s sys 0.028s
I20260808 13:35:08.426983 5771 leader_election.cc:302] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:08.427848 6206 raft_consensus.cc:2781] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:08.428193 6206 raft_consensus.cc:482] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:08.428504 6206 raft_consensus.cc:3037] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:08.440090 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 75740d7986e0439a8677dc14d29ad0e5 (table default.Table_7e2a1be30ada4555aef1f6abf581ccfc [id=fdacc3a340d44e89a022fd9311d68fcc]). Expected version 1 got 0
W20260808 13:35:08.398536 5551 leader_election.cc:341] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:08.446992 5551 leader_election.cc:302] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:08.447956 6185 raft_consensus.cc:2726] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:08.449565 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 54810us
I20260808 13:35:08.455958 5608 tablet_service.cc:1467] Processing CreateTablet for tablet bcc0c6ab755c4a6887dabeeef9364201 (DEFAULT_TABLE table=default.Table_56d20c0c83ee43d7bbad6a06ba148795 [id=16cef8437603476b97a8562edba58a41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:08.460348 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcc0c6ab755c4a6887dabeeef9364201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:08.462953 6212 consensus_queue.cc:1035] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260808 13:35:08.438235 6206 raft_consensus.cc:504] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:08.473701 6211 tablet_replica.cc:330] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:08.474534 6211 raft_consensus.cc:2229] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:08.475008 6211 raft_consensus.cc:2258] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:08.475131 6033 tablet_bootstrap.cc:492] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:08.475651 6033 ts_tablet_manager.cc:1397] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.916s user 0.027s sys 0.040s
I20260808 13:35:08.476814 6211 ts_tablet_manager.cc:1905] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:08.478240 6033 raft_consensus.cc:348] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:08.479060 6033 raft_consensus.cc:374] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:08.490964 6206 leader_election.cc:288] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:08.492575 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_7e2a1be30ada4555aef1f6abf581ccfc" } modify_external_catalogs: true
I20260808 13:35:08.493584 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_7e2a1be30ada4555aef1f6abf581ccfc" } modify_external_catalogs: true
I20260808 13:35:08.495949 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c1f683d6e76f42f086b0442473caa973"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:08.479357 6033 raft_consensus.cc:729] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
W20260808 13:35:08.513271 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1f683d6e76f42f086b0442473caa973 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:08.514838 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edfa66acceea4b54911d78ee16609ef7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:08.515679 5384 raft_consensus.cc:3037] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:08.528885 6131 ts_tablet_manager.cc:1918] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:08.529600 6131 log.cc:1196] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/a1a5a9f860b34797825be7568a0b4ccb
I20260808 13:35:08.540632 5384 raft_consensus.cc:2454] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:08.549409 6154 raft_consensus.cc:482] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:08.550230 6154 raft_consensus.cc:504] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:08.560734 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet c59e1f5f36a2480bbb7288a1449a54de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:08.560928 6154 leader_election.cc:288] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:08.567026 6131 ts_tablet_manager.cc:1939] T a1a5a9f860b34797825be7568a0b4ccb P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:08.577353 6046 tablet_bootstrap.cc:654] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:08.587172 4823 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:37135, user_credentials={real_user=slave}} blocked reactor thread for 96757us
I20260808 13:35:08.594600 6026 tablet_bootstrap.cc:492] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:08.595167 6026 ts_tablet_manager.cc:1397] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.893s user 0.012s sys 0.052s
I20260808 13:35:08.596895 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet a1a5a9f860b34797825be7568a0b4ccb (table default.Table_34602dc423cc4b8b8cc125d6c1ce5452 [id=807c9b982b81419e903344d491fabe09]) successfully deleted
I20260808 13:35:08.604847 6131 tablet_replica.cc:330] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
W20260808 13:35:08.605820 4832 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 66d985de8de040829569d1f5783c6fef consensus state has no leader
I20260808 13:35:08.613181 6131 raft_consensus.cc:2229] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:08.504127 6033 consensus_queue.cc:260] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:08.616360 6033 ts_tablet_manager.cc:1428] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.140s user 0.006s sys 0.000s
I20260808 13:35:08.617388 6033 tablet_bootstrap.cc:492] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:08.604772 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_51a97ceceb424f96bee811933924a07c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:08.630350 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51a97ceceb424f96bee811933924a07c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:08.642038 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8948ce98e7a941f59fa9932fe241638d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:08.647258 5613 consensus_meta.cc:287] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.766s user 0.015s sys 0.014s
I20260808 13:35:08.647933 5771 leader_election.cc:302] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:08.649437 6206 raft_consensus.cc:2781] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:08.650216 6206 raft_consensus.cc:686] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
W20260808 13:35:08.366641 5772 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.674s user 0.000s sys 0.045s
I20260808 13:35:08.651921 6206 consensus_queue.cc:237] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:08.665136 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8948ce98e7a941f59fa9932fe241638d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:08.666129 5881 raft_consensus.cc:2454] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:08.668673 5283 leader_election.cc:302] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:08.670375 6154 raft_consensus.cc:2781] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:35:08.670990 6154 raft_consensus.cc:482] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:08.671586 6154 raft_consensus.cc:3037] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:08.685252 6026 raft_consensus.cc:348] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:08.686005 6026 raft_consensus.cc:374] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:08.686261 6026 raft_consensus.cc:729] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:08.705279 6134 ts_tablet_manager.cc:1918] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:08.614182 6131 raft_consensus.cc:2258] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:08.714203 6131 ts_tablet_manager.cc:1905] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:08.718232 5613 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50600 (request call id 28) took 1210 ms. Trace:
I20260808 13:35:08.701561 6154 raft_consensus.cc:504] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:08.718714 5613 rpcz_store.cc:276] 0808 13:35:07.507879 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:07.508083 (+ 204us) service_pool.cc:227] Handling call
0808 13:35:07.511068 (+ 2985us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:07.511238 (+ 170us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:07.880779 (+369541us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:08.648124 (+767345us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:08.718151 (+ 70027us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":68712}
I20260808 13:35:08.720465 6026 consensus_queue.cc:260] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:08.747094 6033 tablet_bootstrap.cc:654] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:08.752461 4832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5eabc9a0e2b74737ac3e20af0fa63336" } new_table_name: "default.Table_5a0fd24800424aa7b4fa9120704dea8d" modify_external_catalogs: true
I20260808 13:35:08.755354 6046 tablet_bootstrap.cc:492] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:08.755918 6046 ts_tablet_manager.cc:1397] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.453s user 0.015s sys 0.005s
I20260808 13:35:08.756213 4835 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 1 got 0
I20260808 13:35:08.757215 6026 ts_tablet_manager.cc:1428] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.162s user 0.004s sys 0.004s
I20260808 13:35:08.763752 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8948ce98e7a941f59fa9932fe241638d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:08.769784 6026 tablet_bootstrap.cc:492] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:08.769469 4839 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5eabc9a0e2b74737ac3e20af0fa63336 [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 1 got 0
I20260808 13:35:08.721642 6134 log.cc:1196] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/a1a5a9f860b34797825be7568a0b4ccb
I20260808 13:35:08.771041 4839 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]). Expected version 1 got 0
I20260808 13:35:08.771564 5881 raft_consensus.cc:3037] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:08.774127 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8948ce98e7a941f59fa9932fe241638d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:08.777376 6154 leader_election.cc:288] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:08.806840 6134 ts_tablet_manager.cc:1939] T a1a5a9f860b34797825be7568a0b4ccb P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
W20260808 13:35:08.807992 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6d8018bf8ff843d8818fbba9473acf47 consensus state has no leader
I20260808 13:35:08.821651 6211 ts_tablet_manager.cc:1918] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:08.830608 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edfa66acceea4b54911d78ee16609ef7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:08.803447 6046 raft_consensus.cc:348] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:08.841142 5881 raft_consensus.cc:2454] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 2.
I20260808 13:35:08.842792 6046 raft_consensus.cc:374] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:08.843096 5283 leader_election.cc:302] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:08.845783 5880 raft_consensus.cc:1226] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Rejecting Update request from peer 5cd987c3dbdb4168beb73cd7e577c7cb for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260808 13:35:08.835691 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edfa66acceea4b54911d78ee16609ef7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:08.843117 6046 raft_consensus.cc:729] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:08.851897 6046 consensus_queue.cc:260] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:08.853824 6033 tablet_bootstrap.cc:492] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:08.854322 6033 ts_tablet_manager.cc:1397] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.237s user 0.015s sys 0.005s
I20260808 13:35:08.858078 6154 raft_consensus.cc:2781] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:35:08.833019 5644 raft_consensus.cc:2454] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:08.860659 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet a1a5a9f860b34797825be7568a0b4ccb (table default.Table_34602dc423cc4b8b8cc125d6c1ce5452 [id=807c9b982b81419e903344d491fabe09]) successfully deleted
I20260808 13:35:08.868553 6212 consensus_queue.cc:1035] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:08.874745 6211 log.cc:1196] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/a1a5a9f860b34797825be7568a0b4ccb
I20260808 13:35:08.876345 6211 ts_tablet_manager.cc:1939] T a1a5a9f860b34797825be7568a0b4ccb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:08.896286 6212 consensus_queue.cc:1046] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:08.900146 6212 raft_consensus.cc:3032] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Stepping down as leader of term 1
I20260808 13:35:08.900763 6212 raft_consensus.cc:729] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:08.907287 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75740d7986e0439a8677dc14d29ad0e5" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:08.908124 5642 raft_consensus.cc:3037] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:08.911720 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet a1a5a9f860b34797825be7568a0b4ccb (table default.Table_34602dc423cc4b8b8cc125d6c1ce5452 [id=807c9b982b81419e903344d491fabe09]) successfully deleted
I20260808 13:35:08.913352 6211 tablet_replica.cc:330] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:08.915038 6211 raft_consensus.cc:2229] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:08.915903 6211 raft_consensus.cc:2258] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:08.916985 6033 raft_consensus.cc:348] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:08.954743 6154 raft_consensus.cc:686] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 2 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:08.956168 6046 ts_tablet_manager.cc:1428] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.200s user 0.015s sys 0.002s
I20260808 13:35:08.955663 4836 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_ea98c448b3f3473683c7e7e0ac5ae2be" } new_table_name: "default.Table_4159935d23b8404ea7b50d91562d2fc7" modify_external_catalogs: true
I20260808 13:35:08.956993 6154 consensus_queue.cc:237] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:08.960764 6033 raft_consensus.cc:374] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:08.961072 6033 raft_consensus.cc:729] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:08.966415 6046 tablet_bootstrap.cc:492] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:08.966759 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d81d55e146d84d41a68894ad0dd4dc93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:08.969035 6033 consensus_queue.cc:260] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:08.970225 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d81d55e146d84d41a68894ad0dd4dc93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:08.971347 6211 ts_tablet_manager.cc:1905] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:08.980927 6033 ts_tablet_manager.cc:1428] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.126s user 0.008s sys 0.000s
I20260808 13:35:08.997597 6046 tablet_bootstrap.cc:654] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:08.999080 6131 ts_tablet_manager.cc:1918] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:08.999580 6131 log.cc:1196] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/5f18e5b0a4c445a88b7717441edb323f
I20260808 13:35:09.000406 6131 ts_tablet_manager.cc:1939] T 5f18e5b0a4c445a88b7717441edb323f P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:09.008095 6033 tablet_bootstrap.cc:492] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:09.016032 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 5f18e5b0a4c445a88b7717441edb323f (table default.Table_e79e02d050604203b805744df0be2c8c [id=3540defcda954120ba76fe9e4c3869a8]) successfully deleted
I20260808 13:35:09.032953 6026 tablet_bootstrap.cc:654] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:09.034832 6131 tablet_replica.cc:330] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:09.042706 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 344828b16cfc47d5b419f8b569055fa4 (DEFAULT_TABLE table=default.Table_509712ada6da421f9e935ee85e481274 [id=0ddf25cf69f449cf8c7516cb84736b3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.043941 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 344828b16cfc47d5b419f8b569055fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.044454 4842 catalog_manager.cc:5495] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:09.045780 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 3e36f7753d9d4c43afc3c9b253a03846 (table default.Table_703d70b501fa4188a210efda24546a02 [id=e794d9029f6446a996ff766936695c20]). Expected version 1 got 0
I20260808 13:35:09.044394 6033 tablet_bootstrap.cc:654] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:09.052302 5345 tablet_service.cc:1467] Processing CreateTablet for tablet 344828b16cfc47d5b419f8b569055fa4 (DEFAULT_TABLE table=default.Table_509712ada6da421f9e935ee85e481274 [id=0ddf25cf69f449cf8c7516cb84736b3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.053488 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 344828b16cfc47d5b419f8b569055fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.055413 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 5c3535f396f74749a6a28907c6d93abd (DEFAULT_TABLE table=default.Table_c5103bab29d549298c0b01fc0a045e7a [id=d9baa62ea5c249a898586a45f8b4f372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.046573 4842 catalog_manager.cc:5495] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:09.059216 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c3535f396f74749a6a28907c6d93abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.063568 6131 raft_consensus.cc:2229] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:09.064432 6131 raft_consensus.cc:2258] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:09.068358 5351 tablet_service.cc:1467] Processing CreateTablet for tablet a0b29786cb404bc2b01473c3b8390a50 (DEFAULT_TABLE table=default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.069082 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5eabc9a0e2b74737ac3e20af0fa63336 [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 1 got 0
I20260808 13:35:09.070482 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b29786cb404bc2b01473c3b8390a50. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:09.071034 4832 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8948ce98e7a941f59fa9932fe241638d consensus state has no leader
I20260808 13:35:09.071988 5357 tablet_service.cc:1467] Processing CreateTablet for tablet b809b185ee9e4c658a050c41c9abc4ad (DEFAULT_TABLE table=default.Table_1680269ff16246c382d6bccf44f8d52e [id=a15e38df2aee420fb9345cabd03dc842]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.073982 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b809b185ee9e4c658a050c41c9abc4ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.048353 4840 catalog_manager.cc:5495] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:09.079691 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]). Expected version 1 got 0
I20260808 13:35:09.061208 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d052232acdff41f394004ddc3252bbbd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:09.082536 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d052232acdff41f394004ddc3252bbbd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:09.083417 6131 ts_tablet_manager.cc:1905] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:09.056095 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d23d4087551a43ff8652a5ff1418f32b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:09.089754 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d23d4087551a43ff8652a5ff1418f32b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:09.091038 5840 tablet_service.cc:1467] Processing CreateTablet for tablet 5c3535f396f74749a6a28907c6d93abd (DEFAULT_TABLE table=default.Table_c5103bab29d549298c0b01fc0a045e7a [id=d9baa62ea5c249a898586a45f8b4f372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.091603 5836 tablet_service.cc:1467] Processing CreateTablet for tablet b809b185ee9e4c658a050c41c9abc4ad (DEFAULT_TABLE table=default.Table_1680269ff16246c382d6bccf44f8d52e [id=a15e38df2aee420fb9345cabd03dc842]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.093327 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c3535f396f74749a6a28907c6d93abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.099357 5353 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f18e5b0a4c445a88b7717441edb323f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:09.105576 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b809b185ee9e4c658a050c41c9abc4ad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:09.107385 4836 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60f8863bc32e49399940912444bcb972 consensus state has no leader
W20260808 13:35:09.109342 4823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111) [suppressed 8 similar messages]
I20260808 13:35:09.148420 4841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 75740d7986e0439a8677dc14d29ad0e5
I20260808 13:35:09.160450 6046 tablet_bootstrap.cc:492] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:09.160878 6046 ts_tablet_manager.cc:1397] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.198s user 0.009s sys 0.005s
I20260808 13:35:09.162385 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 75740d7986e0439a8677dc14d29ad0e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:09.172761 6046 raft_consensus.cc:348] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:09.173501 6046 raft_consensus.cc:374] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:09.173879 6046 raft_consensus.cc:729] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:09.176743 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 2fa77744bcfb4a5cace730fd9f9fd1ca
I20260808 13:35:09.182849 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:09.184876 5346 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:09.186108 5353 tablet_service.cc:1514] Processing DeleteTablet for tablet 75740d7986e0439a8677dc14d29ad0e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:09.195982 6211 ts_tablet_manager.cc:1918] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:09.196647 6211 log.cc:1196] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/5f18e5b0a4c445a88b7717441edb323f
I20260808 13:35:09.197945 6211 ts_tablet_manager.cc:1939] T 5f18e5b0a4c445a88b7717441edb323f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:09.221586 6046 consensus_queue.cc:260] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:09.225895 6046 ts_tablet_manager.cc:1428] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.065s user 0.006s sys 0.001s
I20260808 13:35:09.252072 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 5f18e5b0a4c445a88b7717441edb323f (table default.Table_e79e02d050604203b805744df0be2c8c [id=3540defcda954120ba76fe9e4c3869a8]) successfully deleted
I20260808 13:35:09.253695 6211 tablet_replica.cc:330] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:09.259893 6211 raft_consensus.cc:2229] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:09.015683 6134 tablet_replica.cc:330] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
W20260808 13:35:09.173908 5551 consensus_peers.cc:487] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:09.269850 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 543debff6da54d26bf62260bf8721ff7 (DEFAULT_TABLE table=default.Table_062151f3d1444efc8a42ba6cd06e19c9 [id=68dee345bd2f4fc1823048b3da8d944f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.271952 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 543debff6da54d26bf62260bf8721ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.265190 6206 raft_consensus.cc:482] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:09.277805 6206 raft_consensus.cc:504] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:09.273862 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_8b1c621ff7f342fba9732b84b40e5292"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:09.282689 6206 leader_election.cc:288] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:09.178758 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 155621us
I20260808 13:35:09.033478 6212 consensus_queue.cc:260] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:09.283732 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b1c621ff7f342fba9732b84b40e5292 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:09.270206 4838 catalog_manager.cc:5495] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 1 to 2, leader changed from 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2) to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 2 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:09.284889 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953a2a65dea7468c8356931cabb6bfc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:09.285303 6211 raft_consensus.cc:2258] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:09.285629 4838 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:09.273329 5838 tablet_service.cc:1467] Processing CreateTablet for tablet a0b29786cb404bc2b01473c3b8390a50 (DEFAULT_TABLE table=default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.274979 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 543debff6da54d26bf62260bf8721ff7 (DEFAULT_TABLE table=default.Table_062151f3d1444efc8a42ba6cd06e19c9 [id=68dee345bd2f4fc1823048b3da8d944f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.290354 4838 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 3e36f7753d9d4c43afc3c9b253a03846 (table default.Table_703d70b501fa4188a210efda24546a02 [id=e794d9029f6446a996ff766936695c20]). Expected version 1 got 0
I20260808 13:35:09.291460 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 543debff6da54d26bf62260bf8721ff7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:09.287210 5771 leader_election.cc:341] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:09.294524 5607 tablet_service.cc:1514] Processing DeleteTablet for tablet 75740d7986e0439a8677dc14d29ad0e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:09.226804 6046 tablet_bootstrap.cc:492] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:09.296252 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 344828b16cfc47d5b419f8b569055fa4 (DEFAULT_TABLE table=default.Table_509712ada6da421f9e935ee85e481274 [id=0ddf25cf69f449cf8c7516cb84736b3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.300081 5609 tablet_service.cc:1467] Processing CreateTablet for tablet 5c3535f396f74749a6a28907c6d93abd (DEFAULT_TABLE table=default.Table_c5103bab29d549298c0b01fc0a045e7a [id=d9baa62ea5c249a898586a45f8b4f372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.289803 6033 tablet_bootstrap.cc:492] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:09.302927 6033 ts_tablet_manager.cc:1397] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.295s user 0.015s sys 0.015s
I20260808 13:35:09.303903 5615 tablet_service.cc:1467] Processing CreateTablet for tablet a0b29786cb404bc2b01473c3b8390a50 (DEFAULT_TABLE table=default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.308028 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b29786cb404bc2b01473c3b8390a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.311857 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_85866a1ec57a4727ae173400b18887d6" } new_table_name: "default.Table_31cc5d89c86849ae876aa34027918bbc" modify_external_catalogs: true
I20260808 13:35:09.316943 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_703d70b501fa4188a210efda24546a02" } new_table_name: "default.Table_3a5f4c4757564738b8e967ff71392b87" modify_external_catalogs: true
W20260808 13:35:09.320385 5606 consensus_meta.cc:287] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.606s user 0.013s sys 0.001s
I20260808 13:35:09.320392 6211 ts_tablet_manager.cc:1905] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:09.329272 5607 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:09.331364 6033 raft_consensus.cc:348] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:09.333029 6033 raft_consensus.cc:374] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:09.333416 6033 raft_consensus.cc:729] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:09.266801 6217 consensus_queue.cc:1035] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.136s
W20260808 13:35:09.337085 5614 consensus_meta.cc:287] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.533s user 0.011s sys 0.000s
I20260808 13:35:09.338109 5614 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50600 (request call id 38) took 1247 ms. Trace:
I20260808 13:35:09.298202 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 344828b16cfc47d5b419f8b569055fa4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:09.365105 6219 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s user 0.031s sys 0.066s
I20260808 13:35:09.334371 6033 consensus_queue.cc:260] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:09.366118 6219 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s user 0.031s sys 0.067s
I20260808 13:35:09.368856 6033 ts_tablet_manager.cc:1428] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.066s user 0.008s sys 0.000s
I20260808 13:35:09.369801 6033 tablet_bootstrap.cc:492] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:09.374706 5611 tablet_service.cc:1467] Processing CreateTablet for tablet b809b185ee9e4c658a050c41c9abc4ad (DEFAULT_TABLE table=default.Table_1680269ff16246c382d6bccf44f8d52e [id=a15e38df2aee420fb9345cabd03dc842]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.377486 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6cb245b3d74d4347843035902b8d49d1" } modify_external_catalogs: true
I20260808 13:35:09.378525 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6cb245b3d74d4347843035902b8d49d1" } modify_external_catalogs: true
I20260808 13:35:09.382550 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_909de5e2ab404a69ba774963323e2102"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:09.384068 6046 tablet_bootstrap.cc:654] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:09.384907 6206 raft_consensus.cc:482] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:09.385982 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_909de5e2ab404a69ba774963323e2102 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:09.385735 6206 raft_consensus.cc:504] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:09.391162 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c140276a5d7438b8c63e02daecd3f80" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:09.391836 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b29786cb404bc2b01473c3b8390a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.392022 5384 raft_consensus.cc:2454] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:09.393036 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c3535f396f74749a6a28907c6d93abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.394170 5771 leader_election.cc:302] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:09.395988 4836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_062151f3d1444efc8a42ba6cd06e19c9" } modify_external_catalogs: true
I20260808 13:35:09.396628 6206 leader_election.cc:288] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:09.396764 4836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_062151f3d1444efc8a42ba6cd06e19c9" } modify_external_catalogs: true
I20260808 13:35:09.397331 6206 raft_consensus.cc:2781] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:09.397845 6206 raft_consensus.cc:482] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:09.398322 6206 raft_consensus.cc:3037] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:09.401434 6131 ts_tablet_manager.cc:1918] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:09.402045 6131 log.cc:1196] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/c59e1f5f36a2480bbb7288a1449a54de
I20260808 13:35:09.460522 6134 raft_consensus.cc:2229] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:09.461833 6134 pending_rounds.cc:62] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: Trying to abort 1 pending ops.
I20260808 13:35:09.462145 6134 pending_rounds.cc:68] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316259259027079168 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "5f18e5b0a4c445a88b7717441edb323f" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 3 new_table_name: "default.Table_e79e02d050604203b805744df0be2c8c" dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" new_extra_config { } }
I20260808 13:35:09.468571 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_fad1a84e78af4cfe8c7ceee0e14a75e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:09.472608 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fad1a84e78af4cfe8c7ceee0e14a75e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:09.326647 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 395153us
I20260808 13:35:09.338425 5614 rpcz_store.cc:276] 0808 13:35:08.328476 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:08.338053 (+ 9577us) service_pool.cc:227] Handling call
0808 13:35:08.352902 (+ 14849us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:08.353020 (+ 118us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:08.803747 (+450727us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:09.337532 (+533785us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:09.338048 (+ 516us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260808 13:35:09.499415 5642 consensus_meta.cc:287] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.591s user 0.008s sys 0.009s
I20260808 13:35:09.506853 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b809b185ee9e4c658a050c41c9abc4ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.513073 5607 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c140276a5d7438b8c63e02daecd3f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:09.462162 6206 raft_consensus.cc:504] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:09.525868 4820 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 47298.3us
I20260808 13:35:09.528169 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 543debff6da54d26bf62260bf8721ff7 (DEFAULT_TABLE table=default.Table_062151f3d1444efc8a42ba6cd06e19c9 [id=68dee345bd2f4fc1823048b3da8d944f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.419131 6131 ts_tablet_manager.cc:1939] T c59e1f5f36a2480bbb7288a1449a54de P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:09.539415 6206 leader_election.cc:288] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:09.540926 6033 tablet_bootstrap.cc:654] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:09.542325 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c140276a5d7438b8c63e02daecd3f80" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:09.543179 5384 raft_consensus.cc:3037] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:09.542701 5385 raft_consensus.cc:1226] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 2 LEADER]: Rejecting Update request from peer 5cd987c3dbdb4168beb73cd7e577c7cb for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260808 13:35:09.551847 6131 tablet_replica.cc:330] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:09.553363 6131 raft_consensus.cc:2229] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:09.540850 6217 raft_consensus.cc:482] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:09.554080 6131 raft_consensus.cc:2258] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:09.554440 6217 raft_consensus.cc:504] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:09.558141 6217 leader_election.cc:288] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:09.559154 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet c59e1f5f36a2480bbb7288a1449a54de (table default.Table_5f0e676e2cf14d31b0439a4fdd39a58f [id=9a99bed19d724319bfaae469dca0ed62]) successfully deleted
W20260808 13:35:09.559921 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 5f18e5b0a4c445a88b7717441edb323f because the tablet was not found. No further retry: Not found: Tablet not found: 5f18e5b0a4c445a88b7717441edb323f
W20260808 13:35:09.559667 5616 consensus_meta.cc:287] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.730s user 0.010s sys 0.007s
I20260808 13:35:09.564983 5384 raft_consensus.cc:2454] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
W20260808 13:35:09.567085 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2fa77744bcfb4a5cace730fd9f9fd1ca already in progress: creating tablet
I20260808 13:35:09.567677 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 543debff6da54d26bf62260bf8721ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.572674 6026 tablet_bootstrap.cc:492] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:09.573338 6026 ts_tablet_manager.cc:1397] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.815s user 0.016s sys 0.008s
I20260808 13:35:09.499995 5642 raft_consensus.cc:2454] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:09.545872 6212 raft_consensus.cc:3037] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:09.580941 6206 consensus_queue.cc:1035] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:35:09.519311 6134 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50600: Aborted: Op aborted
I20260808 13:35:09.587194 6134 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:50600 (request call id 34) took 1582 ms. Trace:
I20260808 13:35:09.587471 5771 leader_election.cc:302] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:35:09.550662 5606 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50600: Illegal state: replica 5cd987c3dbdb4168beb73cd7e577c7cb is not leader of this config: current role FOLLOWER
I20260808 13:35:09.590222 6243 raft_consensus.cc:2781] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:09.591142 5616 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50600 (request call id 35) took 1511 ms. Trace:
I20260808 13:35:09.591805 5616 rpcz_store.cc:276] 0808 13:35:08.080305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:08.090752 (+ 10447us) service_pool.cc:227] Handling call
0808 13:35:08.093095 (+ 2343us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:08.093279 (+ 184us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:08.828912 (+735633us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:09.560850 (+731938us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:09.591062 (+ 30212us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260808 13:35:09.546909 5613 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50600: Illegal state: replica 5cd987c3dbdb4168beb73cd7e577c7cb is not leader of this config: current role FOLLOWER
W20260808 13:35:09.593390 4821 catalog_manager.cc:4920] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): alter failed for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]): Illegal state: replica 5cd987c3dbdb4168beb73cd7e577c7cb is not leader of this config: current role FOLLOWER
I20260808 13:35:09.593967 5608 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50600 (request call id 40) took 1219 ms. Trace:
I20260808 13:35:09.594518 5608 rpcz_store.cc:276] 0808 13:35:08.374372 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:08.383138 (+ 8766us) service_pool.cc:227] Handling call
0808 13:35:08.456664 (+ 73526us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:08.456769 (+ 105us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:08.874101 (+417332us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:09.483424 (+609323us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:09.593869 (+110445us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:09.601227 6131 ts_tablet_manager.cc:1905] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:09.603463 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953a2a65dea7468c8356931cabb6bfc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:09.604307 5644 raft_consensus.cc:2454] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:09.606734 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c140276a5d7438b8c63e02daecd3f80" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:09.607642 5644 raft_consensus.cc:2454] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:09.614146 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c140276a5d7438b8c63e02daecd3f80" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:09.614944 5644 raft_consensus.cc:3037] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:09.616709 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953a2a65dea7468c8356931cabb6bfc9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:09.617622 5881 raft_consensus.cc:2454] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:09.587755 6134 rpcz_store.cc:276] 0808 13:35:08.005247 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:08.010466 (+ 5219us) service_pool.cc:227] Handling call
0808 13:35:09.587055 (+1576589us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:08.159587 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:08.160485 (+ 898us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:08.160564 (+ 79us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:08.160778 (+ 214us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:08.161030 (+ 252us) alter_schema_op.cc:126] START. Timestamp: 1786196108160908
0808 13:35:08.161133 (+ 103us) op_driver.cc:331] REPLICATION: starting
0808 13:35:08.162349 (+ 1216us) log.cc:842] Serialized 173 byte log entry
0808 13:35:09.467178 (+1304829us) op_driver.cc:444] REPLICATION: finished
0808 13:35:09.467256 (+ 78us) op_driver.cc:372] HandleFailure(Aborted: Op aborted)
0808 13:35:09.467297 (+ 41us) alter_schema_op.cc:169] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":94742,"prepare.run_cpu_time_us":3301,"prepare.run_wall_time_us":4514,"replication_time_us":1305888}]]}
I20260808 13:35:09.636732 5772 leader_election.cc:302] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:09.637674 6134 raft_consensus.cc:2258] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:09.637892 5772 raft_consensus.cc:2670] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260808 13:35:09.641326 5772 leader_election.cc:302] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:09.647568 5550 leader_election.cc:302] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:09.655516 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953a2a65dea7468c8356931cabb6bfc9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:09.626130 6243 raft_consensus.cc:686] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:09.660624 6211 ts_tablet_manager.cc:1918] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:09.663308 6211 log.cc:1196] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/75740d7986e0439a8677dc14d29ad0e5
W20260808 13:35:09.650465 6201 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50600: Illegal state: RaftConsensus is not running: State = Stopped
I20260808 13:35:09.664191 6201 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:50600 (request call id 43) took 1285 ms. Trace:
I20260808 13:35:09.671501 6046 tablet_bootstrap.cc:492] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:09.671592 6211 ts_tablet_manager.cc:1939] T 75740d7986e0439a8677dc14d29ad0e5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:09.672029 6046 ts_tablet_manager.cc:1397] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.445s user 0.016s sys 0.030s
I20260808 13:35:09.666503 6206 raft_consensus.cc:2781] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:09.678892 6206 raft_consensus.cc:482] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:09.679708 6206 raft_consensus.cc:3037] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:09.657594 6243 consensus_queue.cc:237] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:09.691403 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 66d985de8de040829569d1f5783c6fef consensus state has no leader
I20260808 13:35:09.697052 6254 tablet.cc:1722] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:09.704294 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 75740d7986e0439a8677dc14d29ad0e5 (table default.Table_7e2a1be30ada4555aef1f6abf581ccfc [id=fdacc3a340d44e89a022fd9311d68fcc]) successfully deleted
I20260808 13:35:09.663335 6026 raft_consensus.cc:348] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:09.706086 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c140276a5d7438b8c63e02daecd3f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:09.705451 6026 raft_consensus.cc:374] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:09.708313 6026 raft_consensus.cc:729] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:09.718676 6217 raft_consensus.cc:2781] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:09.719329 6217 raft_consensus.cc:482] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:09.719905 6217 raft_consensus.cc:3037] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:09.715037 5551 leader_election.cc:341] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:09.656109 6253 tablet.cc:1722] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:09.733215 5882 raft_consensus.cc:1261] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Refusing update from remote peer 946c9d8462644deba5b45fa724f1923c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260808 13:35:09.736400 6154 consensus_queue.cc:1035] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:09.741453 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_0c2610996a984e98b9d1899f23a40bf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:09.744984 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c2610996a984e98b9d1899f23a40bf5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:09.714027 6046 raft_consensus.cc:348] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:09.751099 6046 raft_consensus.cc:374] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:09.751382 6046 raft_consensus.cc:729] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:09.752157 6046 consensus_queue.cc:260] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:09.768019 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2fa77744bcfb4a5cace730fd9f9fd1ca already in progress: creating tablet
I20260808 13:35:09.668540 6201 rpcz_store.cc:276] 0808 13:35:08.379227 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:08.379450 (+ 223us) service_pool.cc:227] Handling call
0808 13:35:09.664099 (+1284649us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:08.382740 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:08.383071 (+ 331us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:09.649691 (+1266620us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:09.649952 (+ 261us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:09.650087 (+ 135us) alter_schema_op.cc:126] START. Timestamp: 1786196109649974
0808 13:35:09.650107 (+ 20us) op_driver.cc:331] REPLICATION: starting
0808 13:35:09.650304 (+ 197us) op_driver.cc:372] HandleFailure(Illegal state: RaftConsensus is not running: State = Stopped)
0808 13:35:09.650349 (+ 45us) alter_schema_op.cc:169] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":154}]]}
I20260808 13:35:09.785949 6046 ts_tablet_manager.cc:1428] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.114s user 0.008s sys 0.000s
I20260808 13:35:09.786965 6046 tablet_bootstrap.cc:492] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:09.811122 6211 tablet_replica.cc:330] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:09.819286 4835 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 1 got 0
I20260808 13:35:09.821039 4835 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 106aef8e805d4a0ebaaf2bdd808bbfab (table default.Table_6db98efa64744617a1464e55349e0f0f [id=984c64fad49d47aca100cfe561c0b780]). Expected version 1 got 0
W20260808 13:35:09.826555 4820 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 52100us
W20260808 13:35:09.833083 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 66d985de8de040829569d1f5783c6fef consensus state has no leader
I20260808 13:35:09.834589 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 81fb3e099f7f4544b3c100070f0d14ef (DEFAULT_TABLE table=default.Table_51a97ceceb424f96bee811933924a07c [id=bdb7331763214cbe82fe544204f8b2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.836761 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81fb3e099f7f4544b3c100070f0d14ef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:09.839591 6201 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50600: Illegal state: RaftConsensus is not running: State = Stopped
I20260808 13:35:09.843433 6201 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:50600 (request call id 45) took 1156 ms. Trace:
I20260808 13:35:09.843566 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 2173e97590be43f9a24f55689d796269 (DEFAULT_TABLE table=default.Table_d052232acdff41f394004ddc3252bbbd [id=6017dfb7d1574e88a452e2d539b1aa03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.870594 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 4625b632d67244fc86e48d739dde707e (DEFAULT_TABLE table=default.Table_c1f683d6e76f42f086b0442473caa973 [id=2f47077a0fd6488ebc33ab8794c0a5b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.884196 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2173e97590be43f9a24f55689d796269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.885515 5607 tablet_service.cc:1467] Processing CreateTablet for tablet cc4dc66771804e0cb8991da63eacba0c (DEFAULT_TABLE table=default.Table_114cb957447a4943806bdadc6ca86549 [id=000c41391410419bb1d4881b34c602fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.891674 5606 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 (DEFAULT_TABLE table=default.Table_34821e686c9b4e988ffdbb94388b2280 [id=ccd192d15e9d42b5ab0739dc606301e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.893230 6026 consensus_queue.cc:260] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:09.914956 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4625b632d67244fc86e48d739dde707e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.919308 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc4dc66771804e0cb8991da63eacba0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.922950 6033 tablet_bootstrap.cc:492] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:09.923413 6033 ts_tablet_manager.cc:1397] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.554s user 0.024s sys 0.017s
I20260808 13:35:09.927623 5642 raft_consensus.cc:1261] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 FOLLOWER]: Refusing update from remote peer 946c9d8462644deba5b45fa724f1923c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260808 13:35:09.928298 5642 pending_rounds.cc:76] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: Aborting all ops after (but not including) 0
I20260808 13:35:09.928656 5642 pending_rounds.cc:97] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260808 13:35:09.928206 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_595f29cce77d4dbf8e1608b573611ad4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:09.929199 5642 raft_consensus.cc:2866] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260808 13:35:09.931656 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_595f29cce77d4dbf8e1608b573611ad4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:09.935611 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd2d0c9ab2c4704a74d06559bb913d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.936939 4835 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_04ef257f28bf43a3b2fec07582b495ac" } new_table_name: "default.Table_ade5bcdd6da94c6284819d3fe37f039f" modify_external_catalogs: true
I20260808 13:35:09.871826 5644 raft_consensus.cc:2454] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:09.953716 6026 ts_tablet_manager.cc:1428] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.345s user 0.023s sys 0.018s
I20260808 13:35:09.954209 5603 tablet_service.cc:1467] Processing CreateTablet for tablet d2ac4c366363468aa62b73e0f1e04532 (DEFAULT_TABLE table=default.Table_d81d55e146d84d41a68894ad0dd4dc93 [id=65666ad40b274ca9a277269157ccbb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.963969 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2ac4c366363468aa62b73e0f1e04532. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:09.970919 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 256382us
I20260808 13:35:09.974802 5644 raft_consensus.cc:3037] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:09.979604 6026 tablet_bootstrap.cc:492] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:09.980154 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 276473us
I20260808 13:35:09.993817 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 2173e97590be43f9a24f55689d796269 (DEFAULT_TABLE table=default.Table_d052232acdff41f394004ddc3252bbbd [id=6017dfb7d1574e88a452e2d539b1aa03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.001286 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 81fb3e099f7f4544b3c100070f0d14ef (DEFAULT_TABLE table=default.Table_51a97ceceb424f96bee811933924a07c [id=bdb7331763214cbe82fe544204f8b2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.005808 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 4625b632d67244fc86e48d739dde707e (DEFAULT_TABLE table=default.Table_c1f683d6e76f42f086b0442473caa973 [id=2f47077a0fd6488ebc33ab8794c0a5b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.007963 5345 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 (DEFAULT_TABLE table=default.Table_34821e686c9b4e988ffdbb94388b2280 [id=ccd192d15e9d42b5ab0739dc606301e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.993868 5353 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c140276a5d7438b8c63e02daecd3f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:05 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:10.011054 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2173e97590be43f9a24f55689d796269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.014921 4838 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_8b1c621ff7f342fba9732b84b40e5292" } new_table_name: "default.Table_92ca3589dbf64a8187402d91d9e889a7" modify_external_catalogs: true
I20260808 13:35:10.017406 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_2b5f163ace454f5e87d04751846435e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:10.021595 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b5f163ace454f5e87d04751846435e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:09.952244 5840 tablet_service.cc:1467] Processing CreateTablet for tablet 4625b632d67244fc86e48d739dde707e (DEFAULT_TABLE table=default.Table_c1f683d6e76f42f086b0442473caa973 [id=2f47077a0fd6488ebc33ab8794c0a5b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.949471 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 81fb3e099f7f4544b3c100070f0d14ef (DEFAULT_TABLE table=default.Table_51a97ceceb424f96bee811933924a07c [id=bdb7331763214cbe82fe544204f8b2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.026571 5842 tablet_service.cc:1467] Processing CreateTablet for tablet cc4dc66771804e0cb8991da63eacba0c (DEFAULT_TABLE table=default.Table_114cb957447a4943806bdadc6ca86549 [id=000c41391410419bb1d4881b34c602fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.027311 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4625b632d67244fc86e48d739dde707e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.953904 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 (DEFAULT_TABLE table=default.Table_34821e686c9b4e988ffdbb94388b2280 [id=ccd192d15e9d42b5ab0739dc606301e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.028836 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc4dc66771804e0cb8991da63eacba0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:09.946813 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 2173e97590be43f9a24f55689d796269 (DEFAULT_TABLE table=default.Table_d052232acdff41f394004ddc3252bbbd [id=6017dfb7d1574e88a452e2d539b1aa03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.987067 6154 consensus_queue.cc:1035] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260808 13:35:10.031531 5346 tablet_service.cc:1467] Processing CreateTablet for tablet cc4dc66771804e0cb8991da63eacba0c (DEFAULT_TABLE table=default.Table_114cb957447a4943806bdadc6ca86549 [id=000c41391410419bb1d4881b34c602fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:09.858393 6201 rpcz_store.cc:276] 0808 13:35:08.687215 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:08.687366 (+ 151us) service_pool.cc:227] Handling call
0808 13:35:09.843335 (+1155969us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:09.838377 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:09.838893 (+ 516us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:09.838965 (+ 72us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:09.839039 (+ 74us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:09.839188 (+ 149us) alter_schema_op.cc:126] START. Timestamp: 1786196109839077
0808 13:35:09.839229 (+ 41us) op_driver.cc:331] REPLICATION: starting
0808 13:35:09.839397 (+ 168us) op_driver.cc:372] HandleFailure(Illegal state: RaftConsensus is not running: State = Stopped)
0808 13:35:09.839491 (+ 94us) alter_schema_op.cc:169] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":1149698}]]}
I20260808 13:35:10.089572 4841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 78d492406c8a4c4186131e02a1559cb4
I20260808 13:35:10.133642 6134 ts_tablet_manager.cc:1905] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:10.163671 4836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 543debff6da54d26bf62260bf8721ff7
I20260808 13:35:10.171342 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d492406c8a4c4186131e02a1559cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:10.177423 4838 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f0c76b819474468baac51365360becfc has no consensus state
I20260808 13:35:10.182931 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 543debff6da54d26bf62260bf8721ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:10.223953 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81fb3e099f7f4544b3c100070f0d14ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.233889 6154 raft_consensus.cc:482] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:10.245867 4835 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5b6e6ce417cc45fd90f5858c30d37a47 consensus state has no leader
I20260808 13:35:10.261670 5845 tablet_service.cc:1467] Processing CreateTablet for tablet d2ac4c366363468aa62b73e0f1e04532 (DEFAULT_TABLE table=default.Table_d81d55e146d84d41a68894ad0dd4dc93 [id=65666ad40b274ca9a277269157ccbb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.259188 6211 raft_consensus.cc:2229] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:10.269881 6211 pending_rounds.cc:62] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Trying to abort 1 pending ops.
I20260808 13:35:10.270216 6211 pending_rounds.cc:68] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259265315926016 op_type: NO_OP noop_request { }
I20260808 13:35:10.255947 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd2d0c9ab2c4704a74d06559bb913d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.273908 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d492406c8a4c4186131e02a1559cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:10.274600 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 543debff6da54d26bf62260bf8721ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:10.279479 6278 raft_consensus.cc:482] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:10.284101 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_f28775e46ccb40e68c299486928fa702"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:10.287211 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f28775e46ccb40e68c299486928fa702 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:09.942517 6206 raft_consensus.cc:504] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:10.294745 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_3e6fd22e0fd24df4b30ca0044f1853bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:10.298297 5609 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50600 (request call id 47) took 1006 ms. Trace:
I20260808 13:35:10.300684 4838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_7ad3b44b6be7476881db975f93533b24" } modify_external_catalogs: true
I20260808 13:35:10.300611 5609 rpcz_store.cc:276] 0808 13:35:09.291905 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.298509 (+ 6604us) service_pool.cc:227] Handling call
0808 13:35:09.301321 (+ 2812us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:09.301441 (+ 120us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:09.825124 (+523683us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:10.297683 (+472559us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:10.298195 (+ 512us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:10.301514 4838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_7ad3b44b6be7476881db975f93533b24" } modify_external_catalogs: true
I20260808 13:35:10.280371 6278 raft_consensus.cc:504] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:10.297525 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e6fd22e0fd24df4b30ca0044f1853bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:10.277292 6033 raft_consensus.cc:348] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:10.304111 6033 raft_consensus.cc:374] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:10.304394 6033 raft_consensus.cc:729] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:10.307824 6033 consensus_queue.cc:260] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:10.286597 6211 raft_consensus.cc:2866] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:35:10.313268 6026 tablet_bootstrap.cc:654] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:10.313652 6033 ts_tablet_manager.cc:1428] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.390s user 0.010s sys 0.001s
I20260808 13:35:10.321221 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2ac4c366363468aa62b73e0f1e04532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.327037 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2173e97590be43f9a24f55689d796269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.329280 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81fb3e099f7f4544b3c100070f0d14ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.214038 6217 raft_consensus.cc:504] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:10.322685 6211 raft_consensus.cc:2258] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:10.337772 6217 leader_election.cc:288] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:10.360610 6206 leader_election.cc:288] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:10.361284 6206 consensus_queue.cc:1035] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:10.375759 6278 leader_election.cc:288] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:10.378087 6253 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:50600 (request call id 44) took 1813 ms. Trace:
I20260808 13:35:10.379503 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f8863bc32e49399940912444bcb972" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:10.381760 5772 leader_election.cc:341] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:10.381474 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_848409f4fa9847b083eac3cb3a0c7afd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:10.384039 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_848409f4fa9847b083eac3cb3a0c7afd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:10.391479 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_509712ada6da421f9e935ee85e481274" } new_table_name: "default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3" modify_external_catalogs: true
I20260808 13:35:10.396175 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953a2a65dea7468c8356931cabb6bfc9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:10.397210 5881 raft_consensus.cc:2379] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:10.400410 6033 tablet_bootstrap.cc:492] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:10.417465 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953a2a65dea7468c8356931cabb6bfc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:10.440424 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd2d0c9ab2c4704a74d06559bb913d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.445747 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d492406c8a4c4186131e02a1559cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:10.446944 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 543debff6da54d26bf62260bf8721ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:10.234548 6154 raft_consensus.cc:504] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:10.430128 4835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6db98efa64744617a1464e55349e0f0f" } modify_external_catalogs: true
I20260808 13:35:10.456364 4835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6db98efa64744617a1464e55349e0f0f" } modify_external_catalogs: true
I20260808 13:35:10.465617 6154 leader_election.cc:288] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:10.468468 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_fc7cb305e35541c781c3e3790cf8a0af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:10.462473 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_51a97ceceb424f96bee811933924a07c" } new_table_name: "default.Table_76c233d2deef48ec8ec2fba9d29f6ecb" modify_external_catalogs: true
I20260808 13:35:10.449924 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953a2a65dea7468c8356931cabb6bfc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:10.475697 5383 raft_consensus.cc:3037] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:10.469556 5644 raft_consensus.cc:2379] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
W20260808 13:35:10.476964 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc7cb305e35541c781c3e3790cf8a0af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:10.472029 5353 tablet_service.cc:1467] Processing CreateTablet for tablet d2ac4c366363468aa62b73e0f1e04532 (DEFAULT_TABLE table=default.Table_d81d55e146d84d41a68894ad0dd4dc93 [id=65666ad40b274ca9a277269157ccbb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.488642 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f8863bc32e49399940912444bcb972" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:10.491406 4823 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c -> Peer 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135): Couldn't send request to peer 87e8b888041b4135af55792050defcb5. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260808 13:35:10.420516 6254 tablet.cc:1722] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260808 13:35:10.516350 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 344828b16cfc47d5b419f8b569055fa4 consensus state has no leader
I20260808 13:35:10.518823 5346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc4dc66771804e0cb8991da63eacba0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.523051 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4625b632d67244fc86e48d739dde707e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.528611 6295 tablet.cc:1722] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:10.548214 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a72fd06a7495444ca9a0f11efc930be6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:10.550829 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2ac4c366363468aa62b73e0f1e04532. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:10.550448 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a72fd06a7495444ca9a0f11efc930be6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:10.560801 5771 leader_election.cc:341] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:10.561522 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 60f8863bc32e49399940912444bcb972 (table default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]). Expected version 1 got 0
I20260808 13:35:10.561403 5771 leader_election.cc:302] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:10.562369 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 3e36f7753d9d4c43afc3c9b253a03846 (table default.Table_3a5f4c4757564738b8e967ff71392b87 [id=e794d9029f6446a996ff766936695c20]). Expected version 2 got 0
W20260808 13:35:10.517566 6281 consensus_queue.cc:1159] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260808 13:35:10.573467 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953a2a65dea7468c8356931cabb6bfc9" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:10.573698 4839 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
I20260808 13:35:10.574884 5881 raft_consensus.cc:2379] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:10.574522 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_38684ebd5f804f82a2132c69d077edfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:10.583796 5606 tablet_service.cc:1467] Processing CreateTablet for tablet 45a26598a69146e1950e9227523fff13 (DEFAULT_TABLE table=default.Table_909de5e2ab404a69ba774963323e2102 [id=e85188ee407c472cbab5ab0ed13d27e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.591773 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a26598a69146e1950e9227523fff13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.590035 4832 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 1 got 0
I20260808 13:35:10.593647 6026 tablet_bootstrap.cc:492] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:10.594076 6026 ts_tablet_manager.cc:1397] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.615s user 0.017s sys 0.010s
W20260808 13:35:10.595063 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38684ebd5f804f82a2132c69d077edfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:10.596731 4832 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:10.609562 6134 ts_tablet_manager.cc:1918] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:10.610086 6134 log.cc:1196] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/5f18e5b0a4c445a88b7717441edb323f
I20260808 13:35:10.612159 6026 raft_consensus.cc:348] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:10.613924 6026 raft_consensus.cc:374] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:10.613754 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 45a26598a69146e1950e9227523fff13 (DEFAULT_TABLE table=default.Table_909de5e2ab404a69ba774963323e2102 [id=e85188ee407c472cbab5ab0ed13d27e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.614537 6026 raft_consensus.cc:729] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:10.621176 4832 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]). Expected version 1 got 0
I20260808 13:35:10.622727 6211 ts_tablet_manager.cc:1905] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:10.631465 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 45a26598a69146e1950e9227523fff13 (DEFAULT_TABLE table=default.Table_909de5e2ab404a69ba774963323e2102 [id=e85188ee407c472cbab5ab0ed13d27e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.633780 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a26598a69146e1950e9227523fff13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.639402 4835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 106aef8e805d4a0ebaaf2bdd808bbfab
I20260808 13:35:10.571795 6243 raft_consensus.cc:2726] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:10.642306 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953a2a65dea7468c8356931cabb6bfc9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:10.644721 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 106aef8e805d4a0ebaaf2bdd808bbfab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:10.647432 6134 ts_tablet_manager.cc:1939] T 5f18e5b0a4c445a88b7717441edb323f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:10.648717 6134 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 41) took 2274 ms. Trace:
I20260808 13:35:10.649386 6134 rpcz_store.cc:276] 0808 13:35:08.374724 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:08.388435 (+ 13711us) service_pool.cc:227] Handling call
0808 13:35:08.913296 (+524861us) ts_tablet_manager.cc:1145] Deleting tablet 5f18e5b0a4c445a88b7717441edb323f
0808 13:35:09.008753 (+ 95457us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:09.845087 (+836334us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b54000a0100. stack: 00007f5a54110033 00007f5a5410ff30 00007f5a53c91534 00000000004db2b4 00000000004db249 00007f5a5b42c361 00007f5a5b4fd417 00007f5a5b4fd726 00007f5a5b4fd54e 00007f5a5b4b5357 00007f5a5c0c2cfa 00007f5a5c0f2e10 00007f5a5c0d9e0d 00007f5a5c0d9db9 00007f5a5c0d9d49 00007f5a5c0d9d11
0808 13:35:10.420137 (+575050us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:10.609317 (+189180us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:10.648628 (+ 39311us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":191543,"spinlock_wait_cycles":54378880,"tablet-delete.queue_time_us":505479}
I20260808 13:35:10.652379 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 5f18e5b0a4c445a88b7717441edb323f (table default.Table_e79e02d050604203b805744df0be2c8c [id=3540defcda954120ba76fe9e4c3869a8]) successfully deleted
I20260808 13:35:10.654556 5551 leader_election.cc:302] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:10.615950 6026 consensus_queue.cc:260] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:10.657757 6259 raft_consensus.cc:2726] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 13:35:10.660145 6026 ts_tablet_manager.cc:1428] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.066s user 0.006s sys 0.005s
I20260808 13:35:10.559060 6046 tablet_bootstrap.cc:654] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:10.664212 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet 106aef8e805d4a0ebaaf2bdd808bbfab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:10.665287 6131 ts_tablet_manager.cc:1918] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:10.666045 6131 log.cc:1196] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/75740d7986e0439a8677dc14d29ad0e5
I20260808 13:35:10.668052 6131 ts_tablet_manager.cc:1939] T 75740d7986e0439a8677dc14d29ad0e5 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:10.638588 5356 tablet_service.cc:1467] Processing CreateTablet for tablet f0c76b819474468baac51365360becfc (DEFAULT_TABLE table=default.Table_92ca3589dbf64a8187402d91d9e889a7 [id=26fb789fff434bb0825daf68d5378013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.691639 5844 tablet_service.cc:1467] Processing CreateTablet for tablet f0c76b819474468baac51365360becfc (DEFAULT_TABLE table=default.Table_92ca3589dbf64a8187402d91d9e889a7 [id=26fb789fff434bb0825daf68d5378013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.692118 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 973a7afdcbf944aab8f11b3fd3995865 (DEFAULT_TABLE table=default.Table_d23d4087551a43ff8652a5ff1418f32b [id=eb20921cee2f48dc86300ce8ce5f52a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.696504 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 973a7afdcbf944aab8f11b3fd3995865 (DEFAULT_TABLE table=default.Table_d23d4087551a43ff8652a5ff1418f32b [id=eb20921cee2f48dc86300ce8ce5f52a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.699258 6026 tablet_bootstrap.cc:492] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:10.702934 6259 raft_consensus.cc:482] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:10.703730 6259 raft_consensus.cc:504] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:10.711346 5607 tablet_service.cc:1467] Processing CreateTablet for tablet 973a7afdcbf944aab8f11b3fd3995865 (DEFAULT_TABLE table=default.Table_d23d4087551a43ff8652a5ff1418f32b [id=eb20921cee2f48dc86300ce8ce5f52a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.713694 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 973a7afdcbf944aab8f11b3fd3995865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.378706 6253 rpcz_store.cc:276] 0808 13:35:08.565238 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:08.566376 (+ 1138us) service_pool.cc:227] Handling call
0808 13:35:10.378020 (+1811644us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:08.760411 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:08.814486 (+ 54075us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:08.814553 (+ 67us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:08.815647 (+ 1094us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:08.815881 (+ 234us) alter_schema_op.cc:126] START. Timestamp: 1786196108786933
0808 13:35:08.815946 (+ 65us) op_driver.cc:331] REPLICATION: starting
0808 13:35:08.820725 (+ 4779us) log.cc:842] Serialized 173 byte log entry
0808 13:35:09.611136 (+790411us) op_driver.cc:444] REPLICATION: finished
0808 13:35:09.655393 (+ 44257us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:09.828303 (+172910us) log.cc:842] Serialized 5 byte log entry
0808 13:35:10.360789 (+532486us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:10.376323 (+ 15534us) log.cc:842] Serialized 14 byte log entry
0808 13:35:10.377759 (+ 1436us) alter_schema_op.cc:71] Released schema lock
0808 13:35:10.377846 (+ 87us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":43863,"prepare.queue_time_us":182767,"prepare.run_cpu_time_us":102456,"prepare.run_wall_time_us":735364,"replication_time_us":795001,"thread_start_us":779479,"threads_started":3,"wal-append.queue_time_us":547290}]]}
I20260808 13:35:10.720480 5603 tablet_service.cc:1467] Processing CreateTablet for tablet f0c76b819474468baac51365360becfc (DEFAULT_TABLE table=default.Table_92ca3589dbf64a8187402d91d9e889a7 [id=26fb789fff434bb0825daf68d5378013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:10.729333 6259 leader_election.cc:288] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:10.710410 6134 tablet_replica.cc:330] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:10.736337 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_fe0257b298fc4b1b933e26b3d8d747bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:10.740432 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe0257b298fc4b1b933e26b3d8d747bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:10.749424 4835 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_848409f4fa9847b083eac3cb3a0c7afd" } new_table_name: "default.Table_d5faf2c508844bc58ff9b879063a3c60" modify_external_catalogs: true
I20260808 13:35:10.761322 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8018bf8ff843d8818fbba9473acf47" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:10.763197 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 106aef8e805d4a0ebaaf2bdd808bbfab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:10.809376 6033 tablet_bootstrap.cc:654] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:10.815770 6134 raft_consensus.cc:2229] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:10.824851 6134 raft_consensus.cc:2258] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:10.829628 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0c76b819474468baac51365360becfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.833734 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 345d13cbde6844079c2d1eb80732bb1f
I20260808 13:35:10.840607 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d13cbde6844079c2d1eb80732bb1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:10.660619 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 231340us
W20260808 13:35:10.872789 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 66d985de8de040829569d1f5783c6fef consensus state has no leader
I20260808 13:35:10.883864 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953a2a65dea7468c8356931cabb6bfc9" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:10.885218 5642 raft_consensus.cc:2379] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:10.897104 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_fc25ff663c1945fabdcce87292fd4050"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:10.903489 4836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fad1a84e78af4cfe8c7ceee0e14a75e5" } modify_external_catalogs: true
I20260808 13:35:10.904388 4836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fad1a84e78af4cfe8c7ceee0e14a75e5" } modify_external_catalogs: true
W20260808 13:35:10.901037 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc25ff663c1945fabdcce87292fd4050 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:10.911412 4833 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 81fb3e099f7f4544b3c100070f0d14ef consensus state has no leader
I20260808 13:35:10.921030 5285 leader_election.cc:302] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:10.880156 5383 raft_consensus.cc:2454] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:10.924489 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8018bf8ff843d8818fbba9473acf47" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:10.925938 6309 tablet.cc:1722] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:10.926952 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0c76b819474468baac51365360becfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.928670 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d13cbde6844079c2d1eb80732bb1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:10.933279 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 973a7afdcbf944aab8f11b3fd3995865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.940919 6134 ts_tablet_manager.cc:1905] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:10.941730 5551 leader_election.cc:341] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:10.967188 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0c76b819474468baac51365360becfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.967964 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 973a7afdcbf944aab8f11b3fd3995865. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:10.972473 4835 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9260370cf8cb46a7b699581e4e8d46db has no consensus state
I20260808 13:35:10.979705 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a26598a69146e1950e9227523fff13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:10.994230 6026 tablet_bootstrap.cc:654] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:11.029238 6033 tablet_bootstrap.cc:492] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:11.031955 6033 ts_tablet_manager.cc:1397] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.632s user 0.009s sys 0.013s
I20260808 13:35:11.045187 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6cd7e5a324184ef3bff0577586e94a29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:11.047389 4835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_0c2610996a984e98b9d1899f23a40bf5" } modify_external_catalogs: true
I20260808 13:35:11.048352 4835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_0c2610996a984e98b9d1899f23a40bf5" } modify_external_catalogs: true
W20260808 13:35:11.054028 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cd7e5a324184ef3bff0577586e94a29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:11.059778 6292 log.cc:924] Time spent T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: Append to log took a long time: real 0.105s user 0.000s sys 0.000s
I20260808 13:35:11.042433 5351 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 57) took 1049 ms. Trace:
I20260808 13:35:11.061260 5351 rpcz_store.cc:276] 0808 13:35:09.992909 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.996405 (+ 3496us) service_pool.cc:227] Handling call
0808 13:35:10.320543 (+324138us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800001898. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c2091af11 00007f5c20a9e851 00007f5c20a9e702 00007f5c29935b09 00007f5c28992730 00007f5c28a74d21 00007f5c28a521e3 00007f5c28a03869 00007f5c270f5c44 00007f5c270f5bd2 00007f5c270f5b01
0808 13:35:10.432035 (+111492us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b5800001898. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c2091af11 00007f5c20a9eb7c 00007f5c20a9e73b 00007f5c29935b09 00007f5c28992730 00007f5c28a74d21 00007f5c28a521e3 00007f5c28a03869 00007f5c270f5c44 00007f5c270f5bd2 00007f5c270f5b01
0808 13:35:10.438929 (+ 6894us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.439104 (+ 175us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:10.926916 (+487812us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:10.937428 (+ 10512us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.042320 (+104892us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":15476992}
I20260808 13:35:11.064800 6300 raft_consensus.cc:2726] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:11.076596 4839 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]). Expected version 1 got 0
I20260808 13:35:11.082463 4839 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 3e36f7753d9d4c43afc3c9b253a03846 (table default.Table_3a5f4c4757564738b8e967ff71392b87 [id=e794d9029f6446a996ff766936695c20]). Expected version 2 got 0
I20260808 13:35:11.103555 6131 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 51) took 1919 ms. Trace:
I20260808 13:35:11.117345 5357 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 55) took 1125 ms. Trace:
W20260808 13:35:11.149091 5771 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 183728us
I20260808 13:35:11.151338 5771 leader_election.cc:302] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:11.152617 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d13cbde6844079c2d1eb80732bb1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:11.154121 5550 leader_election.cc:341] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:11.154547 5550 leader_election.cc:302] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:11.155673 6259 raft_consensus.cc:2726] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:11.156479 6134 ts_tablet_manager.cc:1918] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:11.156913 6134 log.cc:1196] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/75740d7986e0439a8677dc14d29ad0e5
I20260808 13:35:11.156956 4835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 06c59eea10ec45af8e3a0992f9cc74bf
I20260808 13:35:11.157619 6134 ts_tablet_manager.cc:1939] T 75740d7986e0439a8677dc14d29ad0e5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:11.159511 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:11.159545 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:11.162318 5838 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 50) took 1217 ms. Trace:
I20260808 13:35:11.162730 5838 rpcz_store.cc:276] 0808 13:35:09.945025 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.945274 (+ 249us) service_pool.cc:227] Handling call
0808 13:35:10.249462 (+304188us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.257252 (+ 7790us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:10.683486 (+426234us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.161445 (+477959us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.162140 (+ 695us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:11.165724 5836 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 51) took 1217 ms. Trace:
I20260808 13:35:11.166083 5836 rpcz_store.cc:276] 0808 13:35:09.948370 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.948565 (+ 195us) service_pool.cc:227] Handling call
0808 13:35:10.027092 (+ 78527us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.027244 (+ 152us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:10.977942 (+950698us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.165041 (+187099us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.165566 (+ 525us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7383}
I20260808 13:35:11.178333 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
I20260808 13:35:11.179273 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:11.180815 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]). Expected version 1 got 0
I20260808 13:35:11.181506 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 60f8863bc32e49399940912444bcb972 (table default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]). Expected version 1 got 0
I20260808 13:35:11.184646 6026 tablet_bootstrap.cc:492] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:11.186215 6026 ts_tablet_manager.cc:1397] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.488s user 0.012s sys 0.003s
I20260808 13:35:11.186558 4838 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:11.161324 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:11.192576 6134 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 50) took 1898 ms. Trace:
I20260808 13:35:11.193032 6134 rpcz_store.cc:276] 0808 13:35:09.293939 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.294114 (+ 175us) service_pool.cc:227] Handling call
0808 13:35:10.653544 (+1359430us) ts_tablet_manager.cc:1145] Deleting tablet 75740d7986e0439a8677dc14d29ad0e5
0808 13:35:10.653630 (+ 86us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.115328 (+461698us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.156220 (+ 40892us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.192493 (+ 36273us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1358158}
I20260808 13:35:11.194558 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 75740d7986e0439a8677dc14d29ad0e5 (table default.Table_7e2a1be30ada4555aef1f6abf581ccfc [id=fdacc3a340d44e89a022fd9311d68fcc]) successfully deleted
I20260808 13:35:11.194312 6312 tablet.cc:1722] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:11.198333 4836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 024e7f2daaad4f48a79b94344ff35e98
I20260808 13:35:11.200358 6295 tablet.cc:1717] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: Skipping requested alter to schema version 2, tablet already version 2
I20260808 13:35:11.200830 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:11.201568 6295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:50600 (request call id 63) took 1205 ms. Trace:
I20260808 13:35:11.173610 5842 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 54) took 1222 ms. Trace:
I20260808 13:35:11.201946 6295 rpcz_store.cc:276] 0808 13:35:10.019850 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.044278 (+ 24428us) service_pool.cc:227] Handling call
0808 13:35:11.201506 (+1157228us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:10.105985 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:10.106317 (+ 332us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:10.743859 (+637542us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:10.744161 (+ 302us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:10.744310 (+ 149us) alter_schema_op.cc:126] START. Timestamp: 1786196110744236
0808 13:35:10.744348 (+ 38us) op_driver.cc:331] REPLICATION: starting
0808 13:35:10.747409 (+ 3061us) log.cc:842] Serialized 173 byte log entry
0808 13:35:11.199826 (+452417us) op_driver.cc:444] REPLICATION: finished
0808 13:35:11.200185 (+ 359us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:11.201262 (+ 1077us) log.cc:842] Serialized 94 byte log entry
0808 13:35:11.201368 (+ 106us) alter_schema_op.cc:71] Released schema lock
0808 13:35:11.201399 (+ 31us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":197,"prepare.queue_time_us":16504,"prepare.run_cpu_time_us":78130,"prepare.run_wall_time_us":659484,"raft.queue_time_us":127,"raft.run_cpu_time_us":840,"raft.run_wall_time_us":1381,"replication_time_us":455343}]]}
I20260808 13:35:11.205804 5209 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 024e7f2daaad4f48a79b94344ff35e98
I20260808 13:35:11.206369 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:11.208258 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]). Expected version 1 got 0
I20260808 13:35:11.211740 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:11.213268 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:11.214044 6309 tablet.cc:1717] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: Skipping requested alter to schema version 2, tablet already version 2
I20260808 13:35:11.214812 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]). Expected version 1 got 0
I20260808 13:35:11.216353 4838 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_3a5f4c4757564738b8e967ff71392b87 [id=e794d9029f6446a996ff766936695c20] alter complete (version 2)
I20260808 13:35:11.188304 5357 rpcz_store.cc:276] 0808 13:35:09.992188 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.992345 (+ 157us) service_pool.cc:227] Handling call
0808 13:35:10.010219 (+ 17874us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.010429 (+ 210us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:10.866008 (+855579us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.116827 (+250819us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.117282 (+ 455us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:11.103957 6131 rpcz_store.cc:276] 0808 13:35:09.183983 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.185883 (+ 1900us) service_pool.cc:227] Handling call
0808 13:35:09.551596 (+365713us) ts_tablet_manager.cc:1145] Deleting tablet 75740d7986e0439a8677dc14d29ad0e5
0808 13:35:09.551655 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:10.455480 (+903825us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:10.664899 (+209419us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.097528 (+432629us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b5800001898. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c2091af11 00007f5c20a9e851 00007f5c20a9ee82 00007f5c2995e8b9 00007f5c289925d8 00007f5c28a56e42 00007f5c28a86e10 00007f5c28a6de0d 00007f5c28a6ddb9 00007f5c28a6dd49 00007f5c28a6dd11
0808 13:35:11.103484 (+ 5956us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4356480,"tablet-delete.queue_time_us":364906}
I20260808 13:35:11.220388 6309 tablet.cc:1722] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:11.221035 6131 tablet_replica.cc:330] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:11.221800 6131 raft_consensus.cc:2229] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:11.221836 6295 tablet.cc:1722] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:11.222288 6131 raft_consensus.cc:2258] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:11.231281 6134 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 51) took 1936 ms. Trace:
I20260808 13:35:11.231683 6134 rpcz_store.cc:276] 0808 13:35:09.324663 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.328836 (+ 4173us) service_pool.cc:227] Handling call
0808 13:35:11.230862 (+1902026us) ts_tablet_manager.cc:1145] Deleting tablet 2fa77744bcfb4a5cace730fd9f9fd1ca
0808 13:35:11.230913 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.231206 (+ 293us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1900873}
I20260808 13:35:11.234858 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 75740d7986e0439a8677dc14d29ad0e5 (table default.Table_7e2a1be30ada4555aef1f6abf581ccfc [id=fdacc3a340d44e89a022fd9311d68fcc]) successfully deleted
W20260808 13:35:11.235836 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2fa77744bcfb4a5cace730fd9f9fd1ca already in progress: creating tablet
I20260808 13:35:11.236106 6134 tablet_replica.cc:330] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:11.238091 6134 raft_consensus.cc:2229] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:11.238588 6134 raft_consensus.cc:2258] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:11.240818 6134 ts_tablet_manager.cc:1905] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:11.256285 5845 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 55) took 1006 ms. Trace:
I20260808 13:35:11.256645 5845 rpcz_store.cc:276] 0808 13:35:10.256814 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.259497 (+ 2683us) service_pool.cc:227] Handling call
0808 13:35:10.262984 (+ 3487us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.263172 (+ 188us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:10.971130 (+707958us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.255580 (+284450us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.256098 (+ 518us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:11.260294 5209 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 06c59eea10ec45af8e3a0992f9cc74bf
I20260808 13:35:11.261225 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:11.262959 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:11.265702 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:11.245831 6026 raft_consensus.cc:348] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.266758 6026 raft_consensus.cc:374] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:11.266590 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 1947f03e107447bb8e23fe5deaf1ea60 (DEFAULT_TABLE table=default.Table_a72fd06a7495444ca9a0f11efc930be6 [id=a478bcf780ae485ead557bca398e5b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.267050 6026 raft_consensus.cc:729] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:11.269712 6243 raft_consensus.cc:2781] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:11.277026 6026 consensus_queue.cc:260] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.282306 6026 ts_tablet_manager.cc:1428] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.095s user 0.002s sys 0.003s
I20260808 13:35:11.283123 6026 tablet_bootstrap.cc:492] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:11.289592 4838 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a72fd06a7495444ca9a0f11efc930be6" } new_table_name: "default.Table_2dfd25125045402e8caf41f11924eb9a" modify_external_catalogs: true
I20260808 13:35:11.293530 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_97004946b7824885a05a5efd06809df6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:11.295114 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97004946b7824885a05a5efd06809df6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:11.299080 5346 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 59) took 1305 ms. Trace:
I20260808 13:35:11.299530 5346 rpcz_store.cc:276] 0808 13:35:09.993605 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.996492 (+ 2887us) service_pool.cc:227] Handling call
0808 13:35:10.434899 (+438407us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.440004 (+ 5105us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:10.883477 (+443473us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.264347 (+380870us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.298542 (+ 34195us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:11.300942 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1947f03e107447bb8e23fe5deaf1ea60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.301842 6026 tablet_bootstrap.cc:654] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:11.307556 5340 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 56) took 1315 ms. Trace:
I20260808 13:35:11.308163 5340 rpcz_store.cc:276] 0808 13:35:09.992533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.996429 (+ 3896us) service_pool.cc:227] Handling call
0808 13:35:10.222959 (+226530us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.223298 (+ 339us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:10.736476 (+513178us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.274964 (+538488us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.307477 (+ 32513us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:11.316267 6206 raft_consensus.cc:482] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:11.316746 6206 raft_consensus.cc:504] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:11.319223 6285 log.cc:924] Time spent T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
I20260808 13:35:11.320501 5345 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 58) took 1327 ms. Trace:
I20260808 13:35:11.321039 5345 rpcz_store.cc:276] 0808 13:35:09.993236 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.996520 (+ 3284us) service_pool.cc:227] Handling call
0808 13:35:10.432179 (+435659us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.432307 (+ 128us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:11.059576 (+627269us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.277153 (+217577us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.320426 (+ 43273us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:11.324275 6131 ts_tablet_manager.cc:1905] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:11.329557 6033 raft_consensus.cc:348] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.330335 6033 raft_consensus.cc:374] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:11.341423 6134 ts_tablet_manager.cc:1918] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:11.343479 6254 tablet.cc:1717] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Skipping requested alter to schema version 2, tablet already version 2
I20260808 13:35:11.345410 6309 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:46328 (request call id 60) took 1322 ms. Trace:
I20260808 13:35:11.202473 5842 rpcz_store.cc:276] 0808 13:35:09.951566 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.025105 (+ 73539us) service_pool.cc:227] Handling call
0808 13:35:10.027896 (+ 2791us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.028262 (+ 366us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:10.811327 (+783065us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.172818 (+361491us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.173422 (+ 604us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:11.350312 6026 tablet_bootstrap.cc:492] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:11.359424 6026 ts_tablet_manager.cc:1397] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.076s user 0.011s sys 0.000s
I20260808 13:35:11.364418 6026 raft_consensus.cc:348] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.365370 6026 raft_consensus.cc:374] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:11.364868 6033 raft_consensus.cc:729] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:11.365650 6026 raft_consensus.cc:729] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:11.366246 6134 log.cc:1196] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/1c140276a5d7438b8c63e02daecd3f80
I20260808 13:35:11.371191 6134 ts_tablet_manager.cc:1939] T 1c140276a5d7438b8c63e02daecd3f80 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:11.375912 6026 consensus_queue.cc:260] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.380630 6206 leader_election.cc:288] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:11.366192 6033 consensus_queue.cc:260] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.400063 6026 ts_tablet_manager.cc:1428] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.040s user 0.008s sys 0.000s
I20260808 13:35:11.400950 6026 tablet_bootstrap.cc:492] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:11.379246 6134 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 53) took 1868 ms. Trace:
I20260808 13:35:11.403149 6134 rpcz_store.cc:276] 0808 13:35:09.511341 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.512663 (+ 1322us) service_pool.cc:227] Handling call
0808 13:35:11.235941 (+1723278us) ts_tablet_manager.cc:1145] Deleting tablet 1c140276a5d7438b8c63e02daecd3f80
0808 13:35:11.235984 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.296639 (+ 60655us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.341051 (+ 44412us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.379142 (+ 38091us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1722274}
I20260808 13:35:11.406473 6134 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 64) took 1284 ms. Trace:
I20260808 13:35:11.409538 6033 ts_tablet_manager.cc:1428] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.377s user 0.007s sys 0.001s
I20260808 13:35:11.410751 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 1c140276a5d7438b8c63e02daecd3f80 (table default.Table_06f4a6ab47814783a9277ded39511f8f [id=07e09f6780de40928b85f84a463936bb]) successfully deleted
I20260808 13:35:11.416060 6134 rpcz_store.cc:276] 0808 13:35:10.146819 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.169228 (+ 22409us) service_pool.cc:227] Handling call
0808 13:35:11.405716 (+1236488us) ts_tablet_manager.cc:1145] Deleting tablet 78d492406c8a4c4186131e02a1559cb4
0808 13:35:11.405818 (+ 102us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.406386 (+ 568us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1231948}
I20260808 13:35:11.417433 6211 ts_tablet_manager.cc:1918] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:11.418583 6211 log.cc:1196] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/1c140276a5d7438b8c63e02daecd3f80
I20260808 13:35:11.421903 6211 ts_tablet_manager.cc:1939] T 1c140276a5d7438b8c63e02daecd3f80 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:11.430697 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_da97795db35e47d8aed30d142ba853d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:11.436009 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 78d492406c8a4c4186131e02a1559cb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78d492406c8a4c4186131e02a1559cb4 already in progress: creating tablet
I20260808 13:35:11.440366 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_56d20c0c83ee43d7bbad6a06ba148795" } new_table_name: "default.Table_3d3398370b9245fb9b395daaf8dce09d" modify_external_catalogs: true
I20260808 13:35:11.440960 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6f4c6712b74748f58390a05ec941a26a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:11.442729 5606 tablet_service.cc:1467] Processing CreateTablet for tablet 1947f03e107447bb8e23fe5deaf1ea60 (DEFAULT_TABLE table=default.Table_a72fd06a7495444ca9a0f11efc930be6 [id=a478bcf780ae485ead557bca398e5b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.443447 6046 tablet_bootstrap.cc:492] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:11.443943 6046 ts_tablet_manager.cc:1397] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 1.657s user 0.016s sys 0.006s
W20260808 13:35:11.445449 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f4c6712b74748f58390a05ec941a26a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:11.447773 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1947f03e107447bb8e23fe5deaf1ea60. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:11.448532 4838 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1947f03e107447bb8e23fe5deaf1ea60 consensus state has no leader
W20260808 13:35:11.434422 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da97795db35e47d8aed30d142ba853d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:11.345939 6309 rpcz_store.cc:276] 0808 13:35:10.031864 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.032130 (+ 266us) service_pool.cc:227] Handling call
0808 13:35:11.345328 (+1313198us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:10.123235 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:10.123855 (+ 620us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:10.123887 (+ 32us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:10.123943 (+ 56us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:10.124180 (+ 237us) alter_schema_op.cc:126] START. Timestamp: 1786196110124022
0808 13:35:10.124288 (+ 108us) op_driver.cc:331] REPLICATION: starting
0808 13:35:10.222521 (+ 98233us) log.cc:842] Serialized 173 byte log entry
0808 13:35:11.215369 (+992848us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b54000d8520. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c279b9771 00007f5c279c4220 00007f5c279c41c8 00007f5c279c4148 00007f5c279c4103 00007f5c279c338c 00007f5c299ae206 00007f5c299a6808 00007f5c27a0038b 00007f5c27a02e80
0808 13:35:11.216209 (+ 840us) op_driver.cc:444] REPLICATION: finished
0808 13:35:11.220306 (+ 4097us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:11.221537 (+ 1231us) log.cc:842] Serialized 5 byte log entry
0808 13:35:11.246507 (+ 24970us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.344097 (+ 97590us) log.cc:842] Serialized 14 byte log entry
0808 13:35:11.345120 (+ 1023us) alter_schema_op.cc:71] Released schema lock
0808 13:35:11.345178 (+ 58us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3883,"prepare.queue_time_us":66666,"prepare.run_cpu_time_us":19451,"prepare.run_wall_time_us":582573,"raft.queue_time_us":502,"raft.run_cpu_time_us":935,"raft.run_wall_time_us":935,"replication_time_us":1091853,"spinlock_wait_cycles":23838976,"thread_start_us":369929,"threads_started":1,"wal-append.queue_time_us":633347}]]}
I20260808 13:35:11.455252 6134 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 65) took 1287 ms. Trace:
I20260808 13:35:11.455896 6134 rpcz_store.cc:276] 0808 13:35:10.167819 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.172060 (+ 4241us) service_pool.cc:227] Handling call
0808 13:35:11.437022 (+1264962us) ts_tablet_manager.cc:1145] Deleting tablet 543debff6da54d26bf62260bf8721ff7
0808 13:35:11.437073 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.455139 (+ 18066us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1253079}
W20260808 13:35:11.457819 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 543debff6da54d26bf62260bf8721ff7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 543debff6da54d26bf62260bf8721ff7 already in progress: creating tablet
W20260808 13:35:11.458873 6312 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.263s user 0.000s sys 0.008s
I20260808 13:35:11.462028 4839 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4] alter complete (version 2)
I20260808 13:35:11.478000 6046 raft_consensus.cc:348] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.479013 6046 raft_consensus.cc:374] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:11.479332 6046 raft_consensus.cc:729] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:11.479377 6134 tablet_replica.cc:330] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:11.480085 6046 consensus_queue.cc:260] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.480886 6134 raft_consensus.cc:2229] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:11.491219 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddfd9e3f002445738df24ebb0a333440" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:11.492092 5642 raft_consensus.cc:2454] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:11.457154 6243 raft_consensus.cc:686] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:11.498919 6243 consensus_queue.cc:237] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:11.499586 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a5dcf9a6d69a4c1c94e6c32081b818b7" } modify_external_catalogs: true
I20260808 13:35:11.499454 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_0ba9f3241f494f148bd809e66279f480"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:11.500497 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a5dcf9a6d69a4c1c94e6c32081b818b7" } modify_external_catalogs: true
W20260808 13:35:11.503234 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ba9f3241f494f148bd809e66279f480 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:11.503976 6134 raft_consensus.cc:2258] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:11.508354 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d052232acdff41f394004ddc3252bbbd" } modify_external_catalogs: true
I20260808 13:35:11.509104 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d052232acdff41f394004ddc3252bbbd" } modify_external_catalogs: true
I20260808 13:35:11.480352 5840 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 52) took 1529 ms. Trace:
I20260808 13:35:11.510263 5840 rpcz_store.cc:276] 0808 13:35:09.950130 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.950874 (+ 744us) service_pool.cc:227] Handling call
0808 13:35:10.026360 (+ 75486us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.026595 (+ 235us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:10.987918 (+961323us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.478628 (+490710us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.479523 (+ 895us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:11.491398 6033 tablet_bootstrap.cc:492] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:11.514884 5772 leader_election.cc:302] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:11.516933 6328 raft_consensus.cc:2781] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:11.517529 6328 raft_consensus.cc:482] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:11.517261 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 9260370cf8cb46a7b699581e4e8d46db (DEFAULT_TABLE table=default.Table_d5faf2c508844bc58ff9b879063a3c60 [id=b33430fc55db40dd8eb1c9e0396a093b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.520097 5607 tablet_service.cc:1467] Processing CreateTablet for tablet c4c083c8e19d4279af600670f5d1d78d (DEFAULT_TABLE table=default.Table_595f29cce77d4dbf8e1608b573611ad4 [id=4965998f372c444d8e2805605dda7532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.522060 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4c083c8e19d4279af600670f5d1d78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.523226 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9260370cf8cb46a7b699581e4e8d46db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.490268 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]). Expected version 1 got 0
I20260808 13:35:11.532910 6046 ts_tablet_manager.cc:1428] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.089s user 0.009s sys 0.000s
I20260808 13:35:11.533994 6046 tablet_bootstrap.cc:492] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:11.541635 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:11.542234 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:11.518277 6328 raft_consensus.cc:3037] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:11.543354 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:11.504894 6026 tablet_bootstrap.cc:654] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:11.554628 6033 tablet_bootstrap.cc:654] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:11.556141 6134 ts_tablet_manager.cc:1905] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:11.559743 5835 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 53) took 1609 ms. Trace:
I20260808 13:35:11.560263 5835 rpcz_store.cc:276] 0808 13:35:09.950895 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.952342 (+ 1447us) service_pool.cc:227] Handling call
0808 13:35:10.028628 (+ 76286us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:10.028787 (+ 159us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:11.154634 (+1125847us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.559059 (+404425us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:11.559576 (+ 517us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:11.562713 6211 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 49) took 1863 ms. Trace:
I20260808 13:35:11.563133 6211 rpcz_store.cc:276] 0808 13:35:09.699797 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.705748 (+ 5951us) service_pool.cc:227] Handling call
0808 13:35:09.810880 (+105132us) ts_tablet_manager.cc:1145] Deleting tablet 1c140276a5d7438b8c63e02daecd3f80
0808 13:35:09.810951 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:10.269334 (+458383us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54000127d4. stack: 00007f19df247033 00007f19df246f30 00007f19dedc8534 00000000004db2b4 00000000004db249 00007f19e6153471 00007f19e61d6352 00007f19e65ec26c 00007f19e71f9cfa 00007f19e7229e10 00007f19e7210e0d 00007f19e7210db9 00007f19e7210d49 00007f19e7210d11 00007f19e720ff3d 0000000000580784
0808 13:35:11.023078 (+753744us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.416170 (+393092us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.562653 (+146483us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":56572,"spinlock_wait_cycles":1025506048,"tablet-delete.queue_time_us":103934}
I20260808 13:35:11.565043 6211 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 57) took 1291 ms. Trace:
I20260808 13:35:11.578541 6255 raft_consensus.cc:482] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:11.578986 6255 raft_consensus.cc:504] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.580830 6255 leader_election.cc:288] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:11.588243 6211 rpcz_store.cc:276] 0808 13:35:10.274029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.274447 (+ 418us) service_pool.cc:227] Handling call
0808 13:35:11.564587 (+1290140us) ts_tablet_manager.cc:1145] Deleting tablet 543debff6da54d26bf62260bf8721ff7
0808 13:35:11.564657 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.564992 (+ 335us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1285066}
I20260808 13:35:11.588841 5340 tablet_service.cc:1467] Processing CreateTablet for tablet a8e18e51c2e24d3caa3d29e6645e46ae (DEFAULT_TABLE table=default.Table_3e6fd22e0fd24df4b30ca0044f1853bd [id=3c108c9e89424d9a9e11554601cf427c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.589726 6211 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 56) took 1316 ms. Trace:
I20260808 13:35:11.591727 6211 rpcz_store.cc:276] 0808 13:35:10.273588 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.273738 (+ 150us) service_pool.cc:227] Handling call
0808 13:35:11.589340 (+1315602us) ts_tablet_manager.cc:1145] Deleting tablet 78d492406c8a4c4186131e02a1559cb4
0808 13:35:11.589394 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.589674 (+ 280us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1309628}
I20260808 13:35:11.594997 6211 tablet_replica.cc:330] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:11.596326 6211 raft_consensus.cc:2229] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:11.590464 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e18e51c2e24d3caa3d29e6645e46ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.575860 5843 tablet_service.cc:1467] Processing CreateTablet for tablet c4c083c8e19d4279af600670f5d1d78d (DEFAULT_TABLE table=default.Table_595f29cce77d4dbf8e1608b573611ad4 [id=4965998f372c444d8e2805605dda7532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.583070 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f8863bc32e49399940912444bcb972" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:11.608359 5883 raft_consensus.cc:2454] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:11.609980 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4c083c8e19d4279af600670f5d1d78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.611730 5346 tablet_service.cc:1467] Processing CreateTablet for tablet 9260370cf8cb46a7b699581e4e8d46db (DEFAULT_TABLE table=default.Table_d5faf2c508844bc58ff9b879063a3c60 [id=b33430fc55db40dd8eb1c9e0396a093b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.611743 5345 tablet_service.cc:1467] Processing CreateTablet for tablet c4c083c8e19d4279af600670f5d1d78d (DEFAULT_TABLE table=default.Table_595f29cce77d4dbf8e1608b573611ad4 [id=4965998f372c444d8e2805605dda7532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.613334 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4c083c8e19d4279af600670f5d1d78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.586246 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 1947f03e107447bb8e23fe5deaf1ea60 (DEFAULT_TABLE table=default.Table_a72fd06a7495444ca9a0f11efc930be6 [id=a478bcf780ae485ead557bca398e5b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.615281 5346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9260370cf8cb46a7b699581e4e8d46db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.615453 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1947f03e107447bb8e23fe5deaf1ea60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.624312 5840 tablet_service.cc:1467] Processing CreateTablet for tablet 11cd8f12cda94c4eb9a848f002a4020b (DEFAULT_TABLE table=default.Table_2b5f163ace454f5e87d04751846435e8 [id=d5b2a1e2eebe4f1695a98f926a8cb628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.625479 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cd8f12cda94c4eb9a848f002a4020b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.625375 5844 tablet_service.cc:1467] Processing CreateTablet for tablet a8e18e51c2e24d3caa3d29e6645e46ae (DEFAULT_TABLE table=default.Table_3e6fd22e0fd24df4b30ca0044f1853bd [id=3c108c9e89424d9a9e11554601cf427c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.625469 5354 tablet_service.cc:1467] Processing CreateTablet for tablet c8f1618a1d044898bffbc82107e90840 (DEFAULT_TABLE table=default.Table_f28775e46ccb40e68c299486928fa702 [id=1b86c1f240214c88b29348f33a11503d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.626400 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e18e51c2e24d3caa3d29e6645e46ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.626703 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f1618a1d044898bffbc82107e90840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.628633 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 1c140276a5d7438b8c63e02daecd3f80 (table default.Table_06f4a6ab47814783a9277ded39511f8f [id=07e09f6780de40928b85f84a463936bb]) successfully deleted
I20260808 13:35:11.631531 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 11cd8f12cda94c4eb9a848f002a4020b (DEFAULT_TABLE table=default.Table_2b5f163ace454f5e87d04751846435e8 [id=d5b2a1e2eebe4f1695a98f926a8cb628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.631945 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 (DEFAULT_TABLE table=default.Table_fad1a84e78af4cfe8c7ceee0e14a75e5 [id=cda1418c2bf5405abda61222d65589cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.632655 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cd8f12cda94c4eb9a848f002a4020b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.632969 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 024e7f2daaad4f48a79b94344ff35e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.636262 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 (DEFAULT_TABLE table=default.Table_fad1a84e78af4cfe8c7ceee0e14a75e5 [id=cda1418c2bf5405abda61222d65589cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.637360 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 024e7f2daaad4f48a79b94344ff35e98. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:11.630275 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 543debff6da54d26bf62260bf8721ff7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 543debff6da54d26bf62260bf8721ff7 already in progress: creating tablet
W20260808 13:35:11.640154 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 78d492406c8a4c4186131e02a1559cb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78d492406c8a4c4186131e02a1559cb4 already in progress: creating tablet
I20260808 13:35:11.643415 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f8863bc32e49399940912444bcb972" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:11.646072 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddfd9e3f002445738df24ebb0a333440" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:11.647535 5771 leader_election.cc:341] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:11.648522 5836 tablet_service.cc:1467] Processing CreateTablet for tablet c8f1618a1d044898bffbc82107e90840 (DEFAULT_TABLE table=default.Table_f28775e46ccb40e68c299486928fa702 [id=1b86c1f240214c88b29348f33a11503d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.649695 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f1618a1d044898bffbc82107e90840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.655435 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 11cd8f12cda94c4eb9a848f002a4020b (DEFAULT_TABLE table=default.Table_2b5f163ace454f5e87d04751846435e8 [id=d5b2a1e2eebe4f1695a98f926a8cb628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.656680 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cd8f12cda94c4eb9a848f002a4020b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.658577 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 556c57c5b9704b46b142796db2506027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:11.659346 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 (DEFAULT_TABLE table=default.Table_fad1a84e78af4cfe8c7ceee0e14a75e5 [id=cda1418c2bf5405abda61222d65589cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.660491 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 024e7f2daaad4f48a79b94344ff35e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.663412 5609 tablet_service.cc:1467] Processing CreateTablet for tablet c8f1618a1d044898bffbc82107e90840 (DEFAULT_TABLE table=default.Table_f28775e46ccb40e68c299486928fa702 [id=1b86c1f240214c88b29348f33a11503d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.664505 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f1618a1d044898bffbc82107e90840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.669708 4833 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]). Expected version 1 got 0
I20260808 13:35:11.670619 4833 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:11.671146 5550 leader_election.cc:302] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:11.671896 6255 raft_consensus.cc:2781] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:11.672227 6255 raft_consensus.cc:482] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:11.672526 6255 raft_consensus.cc:3037] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:11.671912 5551 leader_election.cc:341] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:11.675065 5613 tablet_service.cc:1467] Processing CreateTablet for tablet a8e18e51c2e24d3caa3d29e6645e46ae (DEFAULT_TABLE table=default.Table_3e6fd22e0fd24df4b30ca0044f1853bd [id=3c108c9e89424d9a9e11554601cf427c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.679281 6134 ts_tablet_manager.cc:1918] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:11.679742 6134 log.cc:1196] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/106aef8e805d4a0ebaaf2bdd808bbfab
I20260808 13:35:11.680531 6134 ts_tablet_manager.cc:1939] T 106aef8e805d4a0ebaaf2bdd808bbfab P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:11.682329 6254 tablet.cc:1722] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:11.680828 6255 raft_consensus.cc:504] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.684074 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e18e51c2e24d3caa3d29e6645e46ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.692314 6255 leader_election.cc:288] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:11.693993 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f8863bc32e49399940912444bcb972" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:11.694566 5883 raft_consensus.cc:3037] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:11.697276 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f8863bc32e49399940912444bcb972" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:11.698736 5551 leader_election.cc:341] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:11.701223 6134 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 67) took 1057 ms. Trace:
I20260808 13:35:11.701534 6134 rpcz_store.cc:276] 0808 13:35:10.643934 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.644471 (+ 537us) service_pool.cc:227] Handling call
0808 13:35:11.479075 (+834604us) ts_tablet_manager.cc:1145] Deleting tablet 106aef8e805d4a0ebaaf2bdd808bbfab
0808 13:35:11.479148 (+ 73us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.653917 (+174769us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.679076 (+ 25159us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.701166 (+ 22090us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":833569}
I20260808 13:35:11.717346 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf (DEFAULT_TABLE table=default.Table_0c2610996a984e98b9d1899f23a40bf5 [id=d158bea8f5fc445e98a01e20b8a53651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.708407 5607 tablet_service.cc:1467] Processing CreateTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf (DEFAULT_TABLE table=default.Table_0c2610996a984e98b9d1899f23a40bf5 [id=d158bea8f5fc445e98a01e20b8a53651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.724090 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c59eea10ec45af8e3a0992f9cc74bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.724056 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 106aef8e805d4a0ebaaf2bdd808bbfab (table default.Table_6db98efa64744617a1464e55349e0f0f [id=984c64fad49d47aca100cfe561c0b780]) successfully deleted
I20260808 13:35:11.727159 6026 tablet_bootstrap.cc:492] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
W20260808 13:35:11.727377 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 345d13cbde6844079c2d1eb80732bb1f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 345d13cbde6844079c2d1eb80732bb1f already in progress: creating tablet
I20260808 13:35:11.727551 6026 ts_tablet_manager.cc:1397] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.327s user 0.012s sys 0.006s
I20260808 13:35:11.731495 6026 raft_consensus.cc:348] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.732239 6026 raft_consensus.cc:374] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:11.732522 6026 raft_consensus.cc:729] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:11.733176 6026 consensus_queue.cc:260] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.735203 6026 ts_tablet_manager.cc:1428] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
W20260808 13:35:11.735386 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bcc0c6ab755c4a6887dabeeef9364201 consensus state has no leader
W20260808 13:35:11.735671 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 06c59eea10ec45af8e3a0992f9cc74bf because the tablet was not found. No further retry: Not found: Tablet not found: 06c59eea10ec45af8e3a0992f9cc74bf
I20260808 13:35:11.736671 6026 tablet_bootstrap.cc:492] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:11.714578 5883 raft_consensus.cc:2454] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
W20260808 13:35:11.751055 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 024e7f2daaad4f48a79b94344ff35e98 because the tablet was not found. No further retry: Not found: Tablet not found: 024e7f2daaad4f48a79b94344ff35e98
I20260808 13:35:11.757418 6131 ts_tablet_manager.cc:1918] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:11.757834 4841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet cfc9a6ec9a86431fb46e50fcaa658d33
I20260808 13:35:11.760943 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 2173e97590be43f9a24f55689d796269
I20260808 13:35:11.763487 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:11.764420 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 2173e97590be43f9a24f55689d796269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:11.765528 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 024e7f2daaad4f48a79b94344ff35e98 because the tablet was not found. No further retry: Not found: Tablet not found: 024e7f2daaad4f48a79b94344ff35e98
W20260808 13:35:11.766369 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 06c59eea10ec45af8e3a0992f9cc74bf because the tablet was not found. No further retry: Not found: Tablet not found: 06c59eea10ec45af8e3a0992f9cc74bf
I20260808 13:35:11.766587 6134 tablet_replica.cc:330] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:11.767302 6134 raft_consensus.cc:2229] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:11.767781 6134 raft_consensus.cc:2258] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:11.769397 6134 ts_tablet_manager.cc:1905] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:11.574396 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 9260370cf8cb46a7b699581e4e8d46db (DEFAULT_TABLE table=default.Table_d5faf2c508844bc58ff9b879063a3c60 [id=b33430fc55db40dd8eb1c9e0396a093b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.719972 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf (DEFAULT_TABLE table=default.Table_0c2610996a984e98b9d1899f23a40bf5 [id=d158bea8f5fc445e98a01e20b8a53651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:11.771345 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c59eea10ec45af8e3a0992f9cc74bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.775116 5550 leader_election.cc:302] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:11.779373 6255 raft_consensus.cc:2781] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:11.782420 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9260370cf8cb46a7b699581e4e8d46db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.760200 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c59eea10ec45af8e3a0992f9cc74bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:11.785415 4839 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d5faf2c508844bc58ff9b879063a3c60" } new_table_name: "default.Table_9948c490176744deaefafd04e2a82ca5" modify_external_catalogs: true
I20260808 13:35:11.761029 6046 tablet_bootstrap.cc:654] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:11.768083 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:11.769642 5353 tablet_service.cc:1514] Processing DeleteTablet for tablet 2173e97590be43f9a24f55689d796269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:11.789335 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_daecadfd962a4dcdaa9fa12e0da68ff0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:11.792999 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (table default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]). Expected version 1 got 0
W20260808 13:35:11.794518 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_daecadfd962a4dcdaa9fa12e0da68ff0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:11.794809 6131 log.cc:1196] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/1c140276a5d7438b8c63e02daecd3f80
I20260808 13:35:11.795900 6131 ts_tablet_manager.cc:1939] T 1c140276a5d7438b8c63e02daecd3f80 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:11.803994 6026 tablet_bootstrap.cc:654] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:11.807556 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:11.810910 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 2173e97590be43f9a24f55689d796269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:11.823166 6131 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 54) took 1842 ms. Trace:
I20260808 13:35:11.823594 6131 rpcz_store.cc:276] 0808 13:35:09.990856 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.993721 (+ 2865us) service_pool.cc:227] Handling call
0808 13:35:11.220865 (+1227144us) ts_tablet_manager.cc:1145] Deleting tablet 1c140276a5d7438b8c63e02daecd3f80
0808 13:35:11.220913 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.523462 (+302549us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.757026 (+233564us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:11.823056 (+ 66030us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1211155}
I20260808 13:35:11.825328 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 1c140276a5d7438b8c63e02daecd3f80 (table default.Table_06f4a6ab47814783a9277ded39511f8f [id=07e09f6780de40928b85f84a463936bb]) successfully deleted
I20260808 13:35:11.825706 6026 tablet_bootstrap.cc:492] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:11.826139 6026 ts_tablet_manager.cc:1397] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.090s user 0.011s sys 0.000s
I20260808 13:35:11.671439 4833 catalog_manager.cc:5495] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:11.827359 4833 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 1 got 0
I20260808 13:35:11.828365 4833 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
I20260808 13:35:11.603184 6328 raft_consensus.cc:504] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.831470 6255 raft_consensus.cc:686] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:11.832413 6255 consensus_queue.cc:237] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.832850 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddfd9e3f002445738df24ebb0a333440" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:11.834331 5771 leader_election.cc:341] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:11.832409 6026 raft_consensus.cc:348] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:11.838213 6026 raft_consensus.cc:374] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:11.838485 6026 raft_consensus.cc:729] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:11.839268 6026 consensus_queue.cc:260] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:11.839862 6211 raft_consensus.cc:2258] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:11.840621 6046 tablet_bootstrap.cc:492] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:11.841099 6046 ts_tablet_manager.cc:1397] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.307s user 0.006s sys 0.006s
I20260808 13:35:11.841429 6026 ts_tablet_manager.cc:1428] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260808 13:35:11.845194 6131 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 62) took 1401 ms. Trace:
I20260808 13:35:11.845600 6131 rpcz_store.cc:276] 0808 13:35:10.443535 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.445386 (+ 1851us) service_pool.cc:227] Handling call
0808 13:35:11.844593 (+1399207us) ts_tablet_manager.cc:1145] Deleting tablet 78d492406c8a4c4186131e02a1559cb4
0808 13:35:11.844689 (+ 96us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.845126 (+ 437us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1398145}
W20260808 13:35:11.847196 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 78d492406c8a4c4186131e02a1559cb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78d492406c8a4c4186131e02a1559cb4 already in progress: creating tablet
I20260808 13:35:11.851377 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddfd9e3f002445738df24ebb0a333440" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:11.851923 5642 raft_consensus.cc:3037] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:11.844059 6046 raft_consensus.cc:348] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.855618 6046 raft_consensus.cc:374] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:11.855973 6046 raft_consensus.cc:729] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:11.859300 5642 raft_consensus.cc:2454] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:11.858338 6026 tablet_bootstrap.cc:492] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:11.861995 5772 leader_election.cc:302] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:11.862870 6243 raft_consensus.cc:2781] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:11.863287 6243 raft_consensus.cc:686] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:11.864059 6243 consensus_queue.cc:237] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.868273 6131 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 63) took 1424 ms. Trace:
I20260808 13:35:11.869617 6046 consensus_queue.cc:260] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:11.874257 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_3d3398370b9245fb9b395daaf8dce09d [id=16cef8437603476b97a8562edba58a41], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bcc0c6ab755c4a6887dabeeef9364201 consensus state has no leader
I20260808 13:35:11.868693 6131 rpcz_store.cc:276] 0808 13:35:10.444262 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.446663 (+ 2401us) service_pool.cc:227] Handling call
0808 13:35:11.867748 (+1421085us) ts_tablet_manager.cc:1145] Deleting tablet 543debff6da54d26bf62260bf8721ff7
0808 13:35:11.867813 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:11.868194 (+ 381us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1420104}
I20260808 13:35:11.881697 6131 tablet_replica.cc:330] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
W20260808 13:35:11.882200 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 543debff6da54d26bf62260bf8721ff7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 543debff6da54d26bf62260bf8721ff7 already in progress: creating tablet
I20260808 13:35:11.882531 6131 raft_consensus.cc:2229] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:11.883159 6131 raft_consensus.cc:2258] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:11.883970 6046 ts_tablet_manager.cc:1428] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.043s user 0.009s sys 0.001s
I20260808 13:35:11.885035 6046 tablet_bootstrap.cc:492] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:11.886126 6033 tablet_bootstrap.cc:492] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:11.886677 6033 ts_tablet_manager.cc:1397] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.396s user 0.012s sys 0.005s
I20260808 13:35:11.897451 6131 ts_tablet_manager.cc:1905] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:11.896145 6328 leader_election.cc:288] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:11.910066 6033 raft_consensus.cc:348] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.910984 6033 raft_consensus.cc:374] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:11.911283 6033 raft_consensus.cc:729] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:11.911933 6033 consensus_queue.cc:260] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.914124 6033 ts_tablet_manager.cc:1428] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260808 13:35:11.916673 6046 tablet_bootstrap.cc:654] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:11.921530 6026 tablet_bootstrap.cc:654] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:11.924516 6211 ts_tablet_manager.cc:1905] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:11.929520 6046 tablet_bootstrap.cc:492] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:11.929913 6046 ts_tablet_manager.cc:1397] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.045s user 0.009s sys 0.000s
I20260808 13:35:11.934782 6046 raft_consensus.cc:348] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.935479 6046 raft_consensus.cc:374] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:11.935714 6046 raft_consensus.cc:729] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:11.936053 6255 raft_consensus.cc:482] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:11.936332 6046 consensus_queue.cc:260] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.936529 6255 raft_consensus.cc:504] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:11.938664 6046 ts_tablet_manager.cc:1428] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260808 13:35:11.938884 6255 leader_election.cc:288] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:11.939508 6046 tablet_bootstrap.cc:492] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:11.939863 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a95712de254c13b3b6dc4fe5cb15f7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:11.941876 4840 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_3d3398370b9245fb9b395daaf8dce09d" } new_table_name: "default.Table_bb876d9598e24abf96366e835a8b2d59" modify_external_catalogs: true
I20260808 13:35:11.937170 6134 ts_tablet_manager.cc:1918] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:11.946369 6134 log.cc:1196] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/556c57c5b9704b46b142796db2506027
I20260808 13:35:11.946856 6046 tablet_bootstrap.cc:654] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:11.946738 5551 leader_election.cc:341] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:11.959932 6134 ts_tablet_manager.cc:1939] T 556c57c5b9704b46b142796db2506027 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:11.955628 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (table default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]). Expected version 1 got 0
I20260808 13:35:11.961287 4832 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
I20260808 13:35:11.971743 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a95712de254c13b3b6dc4fe5cb15f7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:11.976667 5550 leader_election.cc:341] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:11.981119 5550 leader_election.cc:302] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:11.972427 6033 tablet_bootstrap.cc:492] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:11.987112 6255 raft_consensus.cc:2726] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:11.991153 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 556c57c5b9704b46b142796db2506027 (table default.Table_41aa30b0f19f470ea3c1f10a3bf72c73 [id=aa9156d7ba2d452e81c86b72891b6f48]) successfully deleted
W20260808 13:35:11.992053 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet cfc9a6ec9a86431fb46e50fcaa658d33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cfc9a6ec9a86431fb46e50fcaa658d33 already in progress: creating tablet
I20260808 13:35:11.994117 4836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_ade5bcdd6da94c6284819d3fe37f039f" } modify_external_catalogs: true
I20260808 13:35:11.994984 4836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_ade5bcdd6da94c6284819d3fe37f039f" } modify_external_catalogs: true
W20260808 13:35:11.996851 4839 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9260370cf8cb46a7b699581e4e8d46db consensus state has no leader
I20260808 13:35:11.997318 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_b75082dac94348fab668160895573637"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:11.999394 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b75082dac94348fab668160895573637 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:11.999764 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_49299116c37c4b9dbb509dbc8efae85e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:12.001328 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49299116c37c4b9dbb509dbc8efae85e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:11.993700 6033 tablet_bootstrap.cc:654] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:12.020473 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 2173e97590be43f9a24f55689d796269 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2173e97590be43f9a24f55689d796269 already in progress: creating tablet
W20260808 13:35:12.026803 4823 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:37135, user_credentials={real_user=slave}} blocked reactor thread for 112059us
I20260808 13:35:12.028121 4832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6cd7e5a324184ef3bff0577586e94a29" } modify_external_catalogs: true
I20260808 13:35:12.028740 4832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6cd7e5a324184ef3bff0577586e94a29" } modify_external_catalogs: true
I20260808 13:35:12.032261 6026 tablet_bootstrap.cc:492] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:12.031873 4841 catalog_manager.cc:5495] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:12.033043 6026 ts_tablet_manager.cc:1397] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.176s user 0.012s sys 0.001s
I20260808 13:35:12.033278 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]). Expected version 1 got 0
I20260808 13:35:12.042986 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c2b23c8cad8246cfb086daf44bf20256"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:12.045265 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2b23c8cad8246cfb086daf44bf20256 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:12.036322 6026 raft_consensus.cc:348] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.054647 6026 raft_consensus.cc:374] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:12.062748 6026 raft_consensus.cc:729] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:12.063836 6026 consensus_queue.cc:260] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.071627 6026 ts_tablet_manager.cc:1428] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260808 13:35:12.082345 6026 tablet_bootstrap.cc:492] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:12.084569 6131 ts_tablet_manager.cc:1918] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:12.088790 6131 log.cc:1196] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/106aef8e805d4a0ebaaf2bdd808bbfab
I20260808 13:35:12.107761 6131 ts_tablet_manager.cc:1939] T 106aef8e805d4a0ebaaf2bdd808bbfab P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:12.109678 4833 catalog_manager.cc:5495] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:12.111568 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 60f8863bc32e49399940912444bcb972 (table default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]). Expected version 1 got 0
W20260808 13:35:12.122706 4840 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bcc0c6ab755c4a6887dabeeef9364201 consensus state has no leader
I20260808 13:35:12.129485 6131 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 66) took 1486 ms. Trace:
I20260808 13:35:12.140482 6033 tablet_bootstrap.cc:492] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:12.141652 6033 ts_tablet_manager.cc:1397] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.169s user 0.010s sys 0.001s
I20260808 13:35:12.158629 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_55de3cd70f784a7c90b1028236c8118e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:12.159510 6026 tablet_bootstrap.cc:654] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:12.129967 6131 rpcz_store.cc:276] 0808 13:35:10.650466 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.652478 (+ 2012us) service_pool.cc:227] Handling call
0808 13:35:11.881484 (+1229006us) ts_tablet_manager.cc:1145] Deleting tablet 106aef8e805d4a0ebaaf2bdd808bbfab
0808 13:35:11.881549 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:12.022972 (+141423us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:12.084334 (+ 61362us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:12.129358 (+ 45024us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1216556}
W20260808 13:35:12.162634 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55de3cd70f784a7c90b1028236c8118e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:12.163041 6131 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 68) took 1248 ms. Trace:
I20260808 13:35:12.163187 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 106aef8e805d4a0ebaaf2bdd808bbfab (table default.Table_6db98efa64744617a1464e55349e0f0f [id=984c64fad49d47aca100cfe561c0b780]) successfully deleted
I20260808 13:35:12.163451 6131 rpcz_store.cc:276] 0808 13:35:10.923480 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.928432 (+ 4952us) service_pool.cc:227] Handling call
0808 13:35:12.162353 (+1233921us) ts_tablet_manager.cc:1145] Deleting tablet 345d13cbde6844079c2d1eb80732bb1f
0808 13:35:12.162455 (+ 102us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:12.162968 (+ 513us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1233185}
W20260808 13:35:12.167546 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 345d13cbde6844079c2d1eb80732bb1f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 345d13cbde6844079c2d1eb80732bb1f already in progress: creating tablet
I20260808 13:35:12.172338 6131 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 69) took 1011 ms. Trace:
I20260808 13:35:12.172673 6131 rpcz_store.cc:276] 0808 13:35:11.160987 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:11.161167 (+ 180us) service_pool.cc:227] Handling call
0808 13:35:12.171807 (+1010640us) ts_tablet_manager.cc:1145] Deleting tablet 06c59eea10ec45af8e3a0992f9cc74bf
0808 13:35:12.171873 (+ 66us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:12.172240 (+ 367us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":974941}
W20260808 13:35:12.176332 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 06c59eea10ec45af8e3a0992f9cc74bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06c59eea10ec45af8e3a0992f9cc74bf already in progress: creating tablet
I20260808 13:35:12.176257 6033 raft_consensus.cc:348] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.177757 6033 raft_consensus.cc:374] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:12.178005 6033 raft_consensus.cc:729] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
W20260808 13:35:12.180819 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 024e7f2daaad4f48a79b94344ff35e98 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 024e7f2daaad4f48a79b94344ff35e98 already in progress: creating tablet
I20260808 13:35:12.183871 4832 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet ac858759f0d248459073d92b773b455f
W20260808 13:35:12.185097 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 024e7f2daaad4f48a79b94344ff35e98 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 024e7f2daaad4f48a79b94344ff35e98 already in progress: creating tablet
W20260808 13:35:12.186089 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 06c59eea10ec45af8e3a0992f9cc74bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06c59eea10ec45af8e3a0992f9cc74bf already in progress: creating tablet
I20260808 13:35:12.189682 4836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 5b6e6ce417cc45fd90f5858c30d37a47
I20260808 13:35:12.192034 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:12.193264 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:12.193853 5607 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:12.196774 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet cfc9a6ec9a86431fb46e50fcaa658d33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cfc9a6ec9a86431fb46e50fcaa658d33 already in progress: creating tablet
W20260808 13:35:12.197664 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 2173e97590be43f9a24f55689d796269 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2173e97590be43f9a24f55689d796269 already in progress: creating tablet
I20260808 13:35:12.199000 6328 raft_consensus.cc:482] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.199637 6328 raft_consensus.cc:504] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:12.200695 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 5b6e6ce417cc45fd90f5858c30d37a47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5b6e6ce417cc45fd90f5858c30d37a47 already in progress: creating tablet
I20260808 13:35:12.202061 6328 leader_election.cc:288] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:12.206002 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8018bf8ff843d8818fbba9473acf47" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:12.206423 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8018bf8ff843d8818fbba9473acf47" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:12.206750 5642 raft_consensus.cc:2454] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:12.207099 5385 raft_consensus.cc:2454] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:12.208583 5772 leader_election.cc:302] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:35:12.208972 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 5b6e6ce417cc45fd90f5858c30d37a47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5b6e6ce417cc45fd90f5858c30d37a47 already in progress: creating tablet
I20260808 13:35:12.210492 6328 raft_consensus.cc:2781] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:12.211057 6328 raft_consensus.cc:482] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:12.211694 6328 raft_consensus.cc:3037] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.212796 4836 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_daecadfd962a4dcdaa9fa12e0da68ff0" } new_table_name: "default.Table_faaabef2a0ab42648427c33cda679fb7" modify_external_catalogs: true
I20260808 13:35:12.220446 6033 consensus_queue.cc:260] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.223783 6046 tablet_bootstrap.cc:492] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:12.224243 6046 ts_tablet_manager.cc:1397] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.285s user 0.071s sys 0.150s
I20260808 13:35:12.224677 6154 raft_consensus.cc:482] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.225224 6154 raft_consensus.cc:504] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.229449 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddfd9e3f002445738df24ebb0a333440" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:12.230219 5642 raft_consensus.cc:2379] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:12.232096 6026 tablet_bootstrap.cc:492] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:12.233469 6026 ts_tablet_manager.cc:1397] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.151s user 0.013s sys 0.014s
I20260808 13:35:12.233276 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddfd9e3f002445738df24ebb0a333440" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:12.227301 6046 raft_consensus.cc:348] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.234375 6046 raft_consensus.cc:374] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:12.234905 6046 raft_consensus.cc:729] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:12.235589 6046 consensus_queue.cc:260] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.235688 5283 leader_election.cc:302] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:12.236878 6026 raft_consensus.cc:348] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.237594 6026 raft_consensus.cc:374] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:12.237424 6154 leader_election.cc:288] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:12.237851 6026 raft_consensus.cc:729] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:12.249264 6154 raft_consensus.cc:3037] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.238451 6026 consensus_queue.cc:260] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.251488 6033 ts_tablet_manager.cc:1428] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.109s user 0.006s sys 0.001s
I20260808 13:35:12.263543 6026 ts_tablet_manager.cc:1428] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.030s user 0.005s sys 0.005s
I20260808 13:35:12.272428 6328 raft_consensus.cc:504] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.273665 6243 raft_consensus.cc:482] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.275418 6026 tablet_bootstrap.cc:492] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:12.274152 6243 raft_consensus.cc:504] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.279495 6033 tablet_bootstrap.cc:492] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:12.280134 6243 leader_election.cc:288] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:12.281342 6211 ts_tablet_manager.cc:1918] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:12.297315 6211 log.cc:1196] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/106aef8e805d4a0ebaaf2bdd808bbfab
I20260808 13:35:12.298462 6211 ts_tablet_manager.cc:1939] T 106aef8e805d4a0ebaaf2bdd808bbfab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
W20260808 13:35:12.298779 5772 leader_election.cc:341] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 556c57c5b9704b46b142796db2506027
I20260808 13:35:12.303570 6243 consensus_queue.cc:1035] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:12.309248 6211 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 61) took 1594 ms. Trace:
I20260808 13:35:12.312377 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8018bf8ff843d8818fbba9473acf47" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:12.313333 5644 raft_consensus.cc:3037] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.310396 6328 leader_election.cc:288] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:12.318971 6211 rpcz_store.cc:276] 0808 13:35:10.715360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:10.715596 (+ 236us) service_pool.cc:227] Handling call
0808 13:35:11.594787 (+879191us) ts_tablet_manager.cc:1145] Deleting tablet 106aef8e805d4a0ebaaf2bdd808bbfab
0808 13:35:11.594871 (+ 84us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:12.100181 (+505310us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:12.280994 (+180813us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:12.309166 (+ 28172us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":819740}
I20260808 13:35:12.320791 6211 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 62) took 1175 ms. Trace:
I20260808 13:35:12.321723 6026 tablet_bootstrap.cc:654] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:12.321280 6211 rpcz_store.cc:276] 0808 13:35:11.145567 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:11.151949 (+ 6382us) service_pool.cc:227] Handling call
0808 13:35:12.320302 (+1168353us) ts_tablet_manager.cc:1145] Deleting tablet 345d13cbde6844079c2d1eb80732bb1f
0808 13:35:12.320345 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:12.320729 (+ 384us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1166743}
I20260808 13:35:12.321729 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 106aef8e805d4a0ebaaf2bdd808bbfab (table default.Table_6db98efa64744617a1464e55349e0f0f [id=984c64fad49d47aca100cfe561c0b780]) successfully deleted
I20260808 13:35:12.316073 6033 tablet_bootstrap.cc:654] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:12.323364 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 345d13cbde6844079c2d1eb80732bb1f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 345d13cbde6844079c2d1eb80732bb1f already in progress: creating tablet
I20260808 13:35:12.331693 6046 ts_tablet_manager.cc:1428] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.107s user 0.007s sys 0.002s
I20260808 13:35:12.281514 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8018bf8ff843d8818fbba9473acf47" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:12.337361 6046 tablet_bootstrap.cc:492] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:12.342844 6211 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 63) took 1183 ms. Trace:
I20260808 13:35:12.343402 6211 rpcz_store.cc:276] 0808 13:35:11.159197 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:11.159358 (+ 161us) service_pool.cc:227] Handling call
0808 13:35:12.322350 (+1162992us) ts_tablet_manager.cc:1145] Deleting tablet 06c59eea10ec45af8e3a0992f9cc74bf
0808 13:35:12.322400 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:12.342753 (+ 20353us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1162399}
I20260808 13:35:12.355026 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556c57c5b9704b46b142796db2506027" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:12.355643 5383 raft_consensus.cc:2454] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:12.358062 6366 raft_consensus.cc:482] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.358815 6366 raft_consensus.cc:504] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:12.363480 5285 leader_election.cc:341] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 556c57c5b9704b46b142796db2506027
I20260808 13:35:12.347528 5385 raft_consensus.cc:3037] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.377226 5771 leader_election.cc:302] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:12.379874 6243 raft_consensus.cc:2781] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:12.380555 6243 raft_consensus.cc:482] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:12.381074 6243 raft_consensus.cc:3037] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.382903 6325 consensus_queue.cc:1035] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:12.383584 6033 tablet_bootstrap.cc:492] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:12.384140 6033 ts_tablet_manager.cc:1397] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.105s user 0.015s sys 0.003s
I20260808 13:35:12.394976 6033 raft_consensus.cc:348] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.395957 6033 raft_consensus.cc:374] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:12.396240 6033 raft_consensus.cc:729] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:12.402900 6033 consensus_queue.cc:260] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.410320 6033 ts_tablet_manager.cc:1428] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.026s user 0.007s sys 0.000s
I20260808 13:35:12.415627 6033 tablet_bootstrap.cc:492] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:12.418992 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 06c59eea10ec45af8e3a0992f9cc74bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06c59eea10ec45af8e3a0992f9cc74bf already in progress: creating tablet
W20260808 13:35:12.420169 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 024e7f2daaad4f48a79b94344ff35e98 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 024e7f2daaad4f48a79b94344ff35e98 already in progress: creating tablet
W20260808 13:35:12.422756 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 06c59eea10ec45af8e3a0992f9cc74bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06c59eea10ec45af8e3a0992f9cc74bf already in progress: creating tablet
I20260808 13:35:12.362268 6366 leader_election.cc:288] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:12.440255 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]). Expected version 1 got 0
W20260808 13:35:12.447988 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 024e7f2daaad4f48a79b94344ff35e98 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 024e7f2daaad4f48a79b94344ff35e98 already in progress: creating tablet
W20260808 13:35:12.449111 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet cfc9a6ec9a86431fb46e50fcaa658d33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cfc9a6ec9a86431fb46e50fcaa658d33 already in progress: creating tablet
W20260808 13:35:12.450407 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 2173e97590be43f9a24f55689d796269 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2173e97590be43f9a24f55689d796269 already in progress: creating tablet
I20260808 13:35:12.454312 6033 tablet_bootstrap.cc:654] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:12.455415 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 5b6e6ce417cc45fd90f5858c30d37a47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5b6e6ce417cc45fd90f5858c30d37a47 already in progress: creating tablet
I20260808 13:35:12.461915 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 556c57c5b9704b46b142796db2506027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:12.463138 6211 tablet_replica.cc:330] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:12.479763 6046 tablet_bootstrap.cc:654] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:12.482201 6328 raft_consensus.cc:482] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.482867 6328 raft_consensus.cc:504] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.484758 4838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_34821e686c9b4e988ffdbb94388b2280" } modify_external_catalogs: true
I20260808 13:35:12.485486 4838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_34821e686c9b4e988ffdbb94388b2280" } modify_external_catalogs: true
I20260808 13:35:12.491590 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a95712de254c13b3b6dc4fe5cb15f7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:12.492520 5640 raft_consensus.cc:2454] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:12.493541 6033 tablet_bootstrap.cc:492] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:12.494014 6033 ts_tablet_manager.cc:1397] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.079s user 0.013s sys 0.001s
I20260808 13:35:12.494822 5772 leader_election.cc:302] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:12.495602 6325 raft_consensus.cc:2781] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:12.496017 6325 raft_consensus.cc:482] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:12.496507 6325 raft_consensus.cc:3037] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.496140 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fc25ff663c1945fabdcce87292fd4050" } modify_external_catalogs: true
I20260808 13:35:12.496379 6033 raft_consensus.cc:348] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.497187 6033 raft_consensus.cc:374] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:12.497493 6033 raft_consensus.cc:729] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:12.502974 6328 leader_election.cc:288] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:12.497371 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fc25ff663c1945fabdcce87292fd4050" } modify_external_catalogs: true
I20260808 13:35:12.498239 6033 consensus_queue.cc:260] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.515748 6033 ts_tablet_manager.cc:1428] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.021s user 0.008s sys 0.000s
W20260808 13:35:12.519457 4820 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 175782us
I20260808 13:35:12.508476 4832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_97004946b7824885a05a5efd06809df6" } new_table_name: "default.Table_0bde50fe531b4d5a8d76821fde24d7b1" modify_external_catalogs: true
I20260808 13:35:12.509656 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_23396b27c08444c7981e55a66139f6b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:12.534859 6033 tablet_bootstrap.cc:492] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:12.507129 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_8523d4a015194c64b22f66a22b0dcc06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:12.540045 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8523d4a015194c64b22f66a22b0dcc06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:12.540946 5644 raft_consensus.cc:2454] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:12.544071 5772 leader_election.cc:302] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:12.545367 6328 raft_consensus.cc:2781] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:12.546109 6033 tablet_bootstrap.cc:654] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:12.548420 6328 raft_consensus.cc:686] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:12.488862 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a95712de254c13b3b6dc4fe5cb15f7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:12.551772 6243 raft_consensus.cc:504] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.555327 6211 raft_consensus.cc:2229] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:12.556788 6211 raft_consensus.cc:2258] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:12.549575 6328 consensus_queue.cc:237] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.561517 6026 tablet_bootstrap.cc:492] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:12.561965 6026 ts_tablet_manager.cc:1397] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.287s user 0.014s sys 0.034s
I20260808 13:35:12.521287 6325 raft_consensus.cc:504] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.564798 6026 raft_consensus.cc:348] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.565451 6026 raft_consensus.cc:374] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:12.565685 6026 raft_consensus.cc:729] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
W20260808 13:35:12.565776 5772 leader_election.cc:341] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 556c57c5b9704b46b142796db2506027
W20260808 13:35:12.538444 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23396b27c08444c7981e55a66139f6b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:12.569814 6243 leader_election.cc:288] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:12.527390 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 556c57c5b9704b46b142796db2506027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:07 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:12.572793 6131 tablet_replica.cc:330] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:12.573523 6131 raft_consensus.cc:2229] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:12.573964 6131 raft_consensus.cc:2258] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:12.574254 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556c57c5b9704b46b142796db2506027" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:12.575675 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a95712de254c13b3b6dc4fe5cb15f7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:12.575644 6131 ts_tablet_manager.cc:1905] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:12.505995 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 270766us
I20260808 13:35:12.586338 6026 consensus_queue.cc:260] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.605062 6154 raft_consensus.cc:2726] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:12.530516 5385 raft_consensus.cc:2454] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:12.618388 6026 ts_tablet_manager.cc:1428] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.056s user 0.003s sys 0.003s
I20260808 13:35:12.619362 6026 tablet_bootstrap.cc:492] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:12.577364 5771 leader_election.cc:341] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.580010 6033 tablet_bootstrap.cc:492] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:12.634317 6033 ts_tablet_manager.cc:1397] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.100s user 0.008s sys 0.004s
W20260808 13:35:12.634577 4836 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 31fc9fdb5ef543da833f9af6ca5807a9 has no consensus state
W20260808 13:35:12.634532 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 57235.7us
W20260808 13:35:12.636053 5771 leader_election.cc:341] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.636590 5771 leader_election.cc:302] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:12.637552 5771 raft_consensus.cc:2670] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260808 13:35:12.585057 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556c57c5b9704b46b142796db2506027" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:12.641573 6033 raft_consensus.cc:348] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.642309 6033 raft_consensus.cc:374] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260808 13:35:12.638339 5771 leader_election.cc:341] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.642577 6033 raft_consensus.cc:729] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
W20260808 13:35:12.646174 5283 leader_election.cc:341] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.646394 6033 consensus_queue.cc:260] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.646832 5283 leader_election.cc:302] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:12.648069 5283 raft_consensus.cc:2670] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260808 13:35:12.649537 6026 tablet_bootstrap.cc:654] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:12.656301 6211 ts_tablet_manager.cc:1905] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:12.658058 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a95712de254c13b3b6dc4fe5cb15f7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:12.658975 5642 raft_consensus.cc:3037] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.666090 6243 raft_consensus.cc:482] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:12.668076 4832 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6826e4e4db2043b3a4fe9e7a9359cab6 has no consensus state
I20260808 13:35:12.662515 6366 raft_consensus.cc:482] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.673230 6366 raft_consensus.cc:504] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.678869 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f8863bc32e49399940912444bcb972" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:12.669276 6325 leader_election.cc:288] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:12.681670 6366 leader_election.cc:288] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:12.685047 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f8863bc32e49399940912444bcb972" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:12.686075 5883 raft_consensus.cc:2379] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:12.692063 6033 ts_tablet_manager.cc:1428] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.057s user 0.008s sys 0.002s
I20260808 13:35:12.693116 6033 tablet_bootstrap.cc:492] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:12.690290 5283 leader_election.cc:302] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:12.681277 6243 raft_consensus.cc:504] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.702476 6393 raft_consensus.cc:482] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.702934 4841 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet b3ea1e84e40846aab0fbb40503393ab1
I20260808 13:35:12.703033 6393 raft_consensus.cc:504] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.707898 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "328d361c336848689c6906ca6fe2ccc3" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:12.708698 5640 raft_consensus.cc:2454] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:12.710592 5772 leader_election.cc:302] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:12.713027 6391 raft_consensus.cc:2781] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:12.713660 6391 raft_consensus.cc:482] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:12.714017 6255 raft_consensus.cc:482] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.714416 6391 raft_consensus.cc:3037] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.714442 6255 raft_consensus.cc:504] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.716511 6255 leader_election.cc:288] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:12.718267 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7616d81aa924933a9b24d44004725fa" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:12.719223 5883 raft_consensus.cc:2454] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:12.720525 5550 leader_election.cc:302] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:12.721755 6255 raft_consensus.cc:2781] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:12.722230 6255 raft_consensus.cc:482] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:12.722755 6255 raft_consensus.cc:3037] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.724535 6243 leader_election.cc:288] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:12.725066 6390 consensus_queue.cc:1035] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:12.738055 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7616d81aa924933a9b24d44004725fa" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:12.739359 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_339be96b6fc64b40af37836f3d8bedc7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:12.718829 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7616d81aa924933a9b24d44004725fa" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:12.753199 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_339be96b6fc64b40af37836f3d8bedc7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:12.754184 5551 leader_election.cc:341] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.754683 6026 tablet_bootstrap.cc:492] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:12.755182 6026 ts_tablet_manager.cc:1397] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.136s user 0.019s sys 0.006s
I20260808 13:35:12.719766 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7616d81aa924933a9b24d44004725fa" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:12.756738 6255 raft_consensus.cc:504] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.758001 5644 raft_consensus.cc:2379] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:12.758801 6026 raft_consensus.cc:348] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.759573 6026 raft_consensus.cc:374] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:12.760564 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_912bf698cf184d5690eeba3080b2b9bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:12.760737 6154 raft_consensus.cc:3037] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:12.763651 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_912bf698cf184d5690eeba3080b2b9bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:12.765893 5771 leader_election.cc:341] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.759845 6026 raft_consensus.cc:729] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:12.767623 6026 consensus_queue.cc:260] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:12.760399 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 70233.4us
I20260808 13:35:12.772626 6255 leader_election.cc:288] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:12.773488 6255 raft_consensus.cc:482] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.783588 6328 raft_consensus.cc:482] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.784451 6328 raft_consensus.cc:504] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.785895 6026 ts_tablet_manager.cc:1428] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.030s user 0.007s sys 0.001s
I20260808 13:35:12.727706 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "328d361c336848689c6906ca6fe2ccc3" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:12.775203 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7616d81aa924933a9b24d44004725fa" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:12.789991 5551 leader_election.cc:341] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.774112 6255 raft_consensus.cc:504] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.785928 5772 leader_election.cc:302] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:12.773502 6393 leader_election.cc:288] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:12.804251 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7616d81aa924933a9b24d44004725fa" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:12.807034 5883 raft_consensus.cc:3037] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.808887 6255 leader_election.cc:288] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:12.811822 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:12.812824 5882 raft_consensus.cc:2454] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
W20260808 13:35:12.817518 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 56115.3us
I20260808 13:35:12.819096 6026 tablet_bootstrap.cc:492] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:12.837522 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 80110.2us
W20260808 13:35:12.838652 5771 leader_election.cc:341] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.844372 4840 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d23d4087551a43ff8652a5ff1418f32b" } new_table_name: "default.Table_f466df8d38e042cf9c7e3ab344f8aa49" modify_external_catalogs: true
I20260808 13:35:12.845687 6328 leader_election.cc:288] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:12.852837 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:12.835651 6026 tablet_bootstrap.cc:654] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:12.863710 6131 ts_tablet_manager.cc:1918] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:12.850921 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:12.867192 5644 raft_consensus.cc:2454] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:12.823125 5642 raft_consensus.cc:2454] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:12.874485 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_92ca3589dbf64a8187402d91d9e889a7" } new_table_name: "default.Table_9b3bbebdc1274b238c30a45d36cf86f7" modify_external_catalogs: true
I20260808 13:35:12.818130 5550 leader_election.cc:302] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:35:12.875847 5551 leader_election.cc:341] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.877923 6402 raft_consensus.cc:2781] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:12.878335 6402 raft_consensus.cc:482] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260808 13:35:12.878144 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 61048.3us
I20260808 13:35:12.878773 6402 raft_consensus.cc:3037] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.879750 5772 leader_election.cc:302] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:12.881394 5772 leader_election.cc:302] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:12.882529 6255 consensus_queue.cc:1035] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:12.882856 6390 raft_consensus.cc:2781] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:12.883524 6390 raft_consensus.cc:482] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:12.884048 6390 raft_consensus.cc:3037] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:12.883917 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_8d93597641274e5594c62600af306920"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:12.884773 6255 raft_consensus.cc:482] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:12.885236 6255 raft_consensus.cc:504] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:12.885576 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8d93597641274e5594c62600af306920 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:12.887887 6402 raft_consensus.cc:504] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.888350 6255 leader_election.cc:288] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:12.891901 6402 leader_election.cc:288] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:12.892580 6243 raft_consensus.cc:2781] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:12.893117 6243 raft_consensus.cc:686] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:12.894225 6026 tablet_bootstrap.cc:492] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:12.894595 6026 ts_tablet_manager.cc:1397] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.076s user 0.012s sys 0.006s
I20260808 13:35:12.894187 6243 consensus_queue.cc:237] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.897673 6026 raft_consensus.cc:348] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.898388 6026 raft_consensus.cc:374] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:12.905202 6026 raft_consensus.cc:729] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:12.906064 6026 consensus_queue.cc:260] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.908481 6026 ts_tablet_manager.cc:1428] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260808 13:35:12.908689 4836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_3a5f4c4757564738b8e967ff71392b87" } modify_external_catalogs: true
I20260808 13:35:12.909297 4836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_3a5f4c4757564738b8e967ff71392b87" } modify_external_catalogs: true
W20260808 13:35:12.912775 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f0c76b819474468baac51365360becfc consensus state has no leader
I20260808 13:35:12.915254 6026 tablet_bootstrap.cc:492] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:12.916884 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d985de8de040829569d1f5783c6fef" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:12.920008 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:12.929944 6026 tablet_bootstrap.cc:654] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:12.933600 5550 leader_election.cc:341] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.938202 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6008751431314433bd054456abced5f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:12.940188 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6008751431314433bd054456abced5f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:12.942358 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 61129.2us
W20260808 13:35:12.944288 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet f0c76b819474468baac51365360becfc (table default.Table_9b3bbebdc1274b238c30a45d36cf86f7 [id=26fb789fff434bb0825daf68d5378013], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet f0c76b819474468baac51365360becfc consensus state has no leader
I20260808 13:35:12.948179 6026 tablet_bootstrap.cc:492] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:12.948746 6026 ts_tablet_manager.cc:1397] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.011s
I20260808 13:35:12.948928 6131 log.cc:1196] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/556c57c5b9704b46b142796db2506027
I20260808 13:35:12.959075 6033 tablet_bootstrap.cc:654] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:12.878917 6046 tablet_bootstrap.cc:492] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:12.891448 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:12.963507 6026 raft_consensus.cc:348] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.963093 6154 raft_consensus.cc:2726] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:12.964318 6026 raft_consensus.cc:374] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260808 13:35:12.964401 5551 leader_election.cc:341] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.964660 6026 raft_consensus.cc:729] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:12.893509 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d985de8de040829569d1f5783c6fef" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:12.966010 5551 leader_election.cc:341] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.966554 5551 leader_election.cc:302] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:12.967782 6401 raft_consensus.cc:2726] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:12.968645 6046 ts_tablet_manager.cc:1397] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.632s user 0.010s sys 0.011s
I20260808 13:35:12.971062 6046 raft_consensus.cc:348] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:12.967665 4840 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 973a7afdcbf944aab8f11b3fd3995865 consensus state has no leader
W20260808 13:35:12.972442 6406 log.cc:924] Time spent T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.066s user 0.001s sys 0.000s
I20260808 13:35:12.899227 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:12.977022 6046 raft_consensus.cc:374] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260808 13:35:12.977497 5771 leader_election.cc:341] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:12.979053 6046 raft_consensus.cc:729] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:12.981338 6046 consensus_queue.cc:260] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:12.985276 4836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 3e36f7753d9d4c43afc3c9b253a03846
I20260808 13:35:12.989635 5607 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e36f7753d9d4c43afc3c9b253a03846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:12.989569 6390 raft_consensus.cc:504] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.992161 5883 raft_consensus.cc:2454] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:12.992726 6393 raft_consensus.cc:2726] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:12.993193 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e36f7753d9d4c43afc3c9b253a03846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:12.980473 6391 raft_consensus.cc:504] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.999533 5550 leader_election.cc:302] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:13.002316 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "328d361c336848689c6906ca6fe2ccc3" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:13.003002 5642 raft_consensus.cc:3037] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:13.003281 6401 raft_consensus.cc:2781] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:13.010128 5642 raft_consensus.cc:2454] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:13.010288 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_4a3f8ae1263244bd80f5eb36424e85e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:13.011294 5772 leader_election.cc:302] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:13.012157 6393 raft_consensus.cc:2781] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
W20260808 13:35:13.012176 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a3f8ae1263244bd80f5eb36424e85e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:13.012871 6393 raft_consensus.cc:686] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:13.014503 6046 ts_tablet_manager.cc:1428] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.045s user 0.005s sys 0.001s
I20260808 13:35:12.949765 6131 ts_tablet_manager.cc:1939] T 556c57c5b9704b46b142796db2506027 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:13.014032 6393 consensus_queue.cc:237] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:12.997622 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e36f7753d9d4c43afc3c9b253a03846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:13.002197 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "328d361c336848689c6906ca6fe2ccc3" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:13.018443 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 556c57c5b9704b46b142796db2506027 (table default.Table_41aa30b0f19f470ea3c1f10a3bf72c73 [id=aa9156d7ba2d452e81c86b72891b6f48]) successfully deleted
W20260808 13:35:13.018719 5771 leader_election.cc:341] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:13.019511 6131 tablet_replica.cc:330] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:13.020200 6211 ts_tablet_manager.cc:1918] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:13.020546 6131 raft_consensus.cc:2229] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:13.021064 6211 log.cc:1196] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/556c57c5b9704b46b142796db2506027
I20260808 13:35:13.021135 6131 raft_consensus.cc:2258] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:13.022143 6401 raft_consensus.cc:686] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:12.965668 6026 consensus_queue.cc:260] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.024137 6401 consensus_queue.cc:237] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.024137 6026 ts_tablet_manager.cc:1428] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.075s user 0.006s sys 0.002s
I20260808 13:35:13.024854 5880 raft_consensus.cc:2379] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:13.033281 6026 tablet_bootstrap.cc:492] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:13.042196 6211 ts_tablet_manager.cc:1939] T 556c57c5b9704b46b142796db2506027 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:13.043726 5550 leader_election.cc:302] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:13.044865 6255 raft_consensus.cc:2726] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 13:35:13.051446 6419 tablet_replica.cc:330] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:13.071782 6131 ts_tablet_manager.cc:1905] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:13.075093 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:13.050661 6391 leader_election.cc:288] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:13.061884 6243 consensus_queue.cc:1035] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:13.078768 6046 tablet_bootstrap.cc:492] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:13.082736 6390 leader_election.cc:288] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:13.065053 6033 tablet_bootstrap.cc:492] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
W20260808 13:35:13.086858 5771 leader_election.cc:341] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:13.088267 6033 ts_tablet_manager.cc:1397] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.395s user 0.013s sys 0.014s
I20260808 13:35:13.091442 6033 raft_consensus.cc:348] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.062052 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:13.097843 5642 raft_consensus.cc:2379] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:13.098527 6419 raft_consensus.cc:2229] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:13.099530 6419 raft_consensus.cc:2258] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:13.101576 6419 ts_tablet_manager.cc:1905] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:13.112418 4835 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 1 got 0
I20260808 13:35:13.113147 4835 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (table default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]). Expected version 1 got 0
I20260808 13:35:13.115149 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 556c57c5b9704b46b142796db2506027 (table default.Table_41aa30b0f19f470ea3c1f10a3bf72c73 [id=aa9156d7ba2d452e81c86b72891b6f48]) successfully deleted
I20260808 13:35:13.118305 6046 tablet_bootstrap.cc:654] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:13.121629 4839 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c2b23c8cad8246cfb086daf44bf20256" } new_table_name: "default.Table_7145b6a6927345f4b8650878e403d7d3" modify_external_catalogs: true
I20260808 13:35:13.099758 5772 leader_election.cc:302] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:13.123481 6390 raft_consensus.cc:2726] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 13:35:13.106805 6211 tablet_replica.cc:330] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:13.124738 6211 raft_consensus.cc:2229] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:13.125275 6211 raft_consensus.cc:2258] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:13.125988 5844 tablet_service.cc:1467] Processing CreateTablet for tablet a541ec35b1a44109866fabc953f14cc6 (DEFAULT_TABLE table=default.Table_0ba9f3241f494f148bd809e66279f480 [id=5b1fc9924efa44b8807de40565cdd419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.126773 6033 raft_consensus.cc:374] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:13.127038 6033 raft_consensus.cc:729] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:13.127696 6033 consensus_queue.cc:260] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.132470 6033 ts_tablet_manager.cc:1428] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.044s user 0.001s sys 0.005s
I20260808 13:35:13.102044 6255 consensus_queue.cc:1035] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:13.132598 5609 tablet_service.cc:1467] Processing CreateTablet for tablet 896389941179456d9d0ed3cd14726f11 (DEFAULT_TABLE table=default.Table_38684ebd5f804f82a2132c69d077edfe [id=d7c7a7f08fa543aaa0bca2e996d1ee8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.134112 5603 tablet_service.cc:1467] Processing CreateTablet for tablet e992a0294c404cfb9541b57782a05fc9 (DEFAULT_TABLE table=default.Table_6f4c6712b74748f58390a05ec941a26a [id=2b4fc1c958044df7939df70bc136d956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.135350 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 261e45848f424a349b0c5290954c5bc6 (DEFAULT_TABLE table=default.Table_da97795db35e47d8aed30d142ba853d2 [id=5d87520fa77a4254878c3f2d49f6592b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.135619 6211 ts_tablet_manager.cc:1905] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:13.137171 5613 tablet_service.cc:1467] Processing CreateTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e (DEFAULT_TABLE table=default.Table_fc7cb305e35541c781c3e3790cf8a0af [id=b885c5ab382c45809d6f896ebf8771bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.139488 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a541ec35b1a44109866fabc953f14cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.139719 5606 tablet_service.cc:1467] Processing CreateTablet for tablet b8ea98afef6841fead641b314ee53a6e (DEFAULT_TABLE table=default.Table_fe0257b298fc4b1b933e26b3d8d747bd [id=4694a5eaa9524750b04d6e92ef4720ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.153828 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 261e45848f424a349b0c5290954c5bc6 (DEFAULT_TABLE table=default.Table_da97795db35e47d8aed30d142ba853d2 [id=5d87520fa77a4254878c3f2d49f6592b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.154699 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 (DEFAULT_TABLE table=default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.155303 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261e45848f424a349b0c5290954c5bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.156487 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6826e4e4db2043b3a4fe9e7a9359cab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.159785 5840 tablet_service.cc:1467] Processing CreateTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e (DEFAULT_TABLE table=default.Table_fc7cb305e35541c781c3e3790cf8a0af [id=b885c5ab382c45809d6f896ebf8771bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.159801 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 896389941179456d9d0ed3cd14726f11 (DEFAULT_TABLE table=default.Table_38684ebd5f804f82a2132c69d077edfe [id=d7c7a7f08fa543aaa0bca2e996d1ee8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.117376 5607 tablet_service.cc:1467] Processing CreateTablet for tablet a541ec35b1a44109866fabc953f14cc6 (DEFAULT_TABLE table=default.Table_0ba9f3241f494f148bd809e66279f480 [id=5b1fc9924efa44b8807de40565cdd419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.161538 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3682c9c656c4556bc13ae3aeddd7c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.163686 5836 tablet_service.cc:1467] Processing CreateTablet for tablet b8ea98afef6841fead641b314ee53a6e (DEFAULT_TABLE table=default.Table_fe0257b298fc4b1b933e26b3d8d747bd [id=4694a5eaa9524750b04d6e92ef4720ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.164986 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8ea98afef6841fead641b314ee53a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.167109 5842 tablet_service.cc:1467] Processing CreateTablet for tablet e992a0294c404cfb9541b57782a05fc9 (DEFAULT_TABLE table=default.Table_6f4c6712b74748f58390a05ec941a26a [id=2b4fc1c958044df7939df70bc136d956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.168133 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e992a0294c404cfb9541b57782a05fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.168648 6033 tablet_bootstrap.cc:492] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:13.174012 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896389941179456d9d0ed3cd14726f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.180207 4832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_9948c490176744deaefafd04e2a82ca5" } new_table_name: "default.Table_4026b16c74ac4327ac4ba5f178bfcaf6" modify_external_catalogs: true
I20260808 13:35:13.181542 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_eeada3d5a8ba4dc4b96d74955c23f2bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:13.183153 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eeada3d5a8ba4dc4b96d74955c23f2bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:13.134400 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896389941179456d9d0ed3cd14726f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.187055 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e992a0294c404cfb9541b57782a05fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.188458 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261e45848f424a349b0c5290954c5bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.189155 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8ea98afef6841fead641b314ee53a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.189134 5355 tablet_service.cc:1467] Processing CreateTablet for tablet a541ec35b1a44109866fabc953f14cc6 (DEFAULT_TABLE table=default.Table_0ba9f3241f494f148bd809e66279f480 [id=5b1fc9924efa44b8807de40565cdd419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.189977 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a541ec35b1a44109866fabc953f14cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.191732 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a541ec35b1a44109866fabc953f14cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.193467 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_0ba9f3241f494f148bd809e66279f480" } modify_external_catalogs: true
I20260808 13:35:13.196028 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 (DEFAULT_TABLE table=default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.197645 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6826e4e4db2043b3a4fe9e7a9359cab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.200528 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 896389941179456d9d0ed3cd14726f11 (DEFAULT_TABLE table=default.Table_38684ebd5f804f82a2132c69d077edfe [id=d7c7a7f08fa543aaa0bca2e996d1ee8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.201033 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 261e45848f424a349b0c5290954c5bc6 (DEFAULT_TABLE table=default.Table_da97795db35e47d8aed30d142ba853d2 [id=5d87520fa77a4254878c3f2d49f6592b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.202121 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896389941179456d9d0ed3cd14726f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.202943 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261e45848f424a349b0c5290954c5bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.118908 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 60f8863bc32e49399940912444bcb972 (table default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]). Expected version 1 got 0
I20260808 13:35:13.205303 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 1 got 0
I20260808 13:35:13.210882 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 1 got 0
I20260808 13:35:13.211803 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 2 got 0
I20260808 13:35:13.227769 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3682c9c656c4556bc13ae3aeddd7c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.228214 5354 tablet_service.cc:1467] Processing CreateTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e (DEFAULT_TABLE table=default.Table_fc7cb305e35541c781c3e3790cf8a0af [id=b885c5ab382c45809d6f896ebf8771bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.229915 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3682c9c656c4556bc13ae3aeddd7c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.236531 5346 tablet_service.cc:1467] Processing CreateTablet for tablet b8ea98afef6841fead641b314ee53a6e (DEFAULT_TABLE table=default.Table_fe0257b298fc4b1b933e26b3d8d747bd [id=4694a5eaa9524750b04d6e92ef4720ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.238196 5346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8ea98afef6841fead641b314ee53a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.240075 5353 tablet_service.cc:1467] Processing CreateTablet for tablet e992a0294c404cfb9541b57782a05fc9 (DEFAULT_TABLE table=default.Table_6f4c6712b74748f58390a05ec941a26a [id=2b4fc1c958044df7939df70bc136d956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.250082 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.60:37135
--webserver_interface=127.4.170.60
--webserver_port=45565
--builtin_ntp_servers=127.4.170.20:42231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260808 13:35:13.255595 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 20) took 779 ms (client timeout 1000 ms). Trace:
I20260808 13:35:13.257400 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e992a0294c404cfb9541b57782a05fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.261588 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d492406c8a4c4186131e02a1559cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:13.124430 6026 tablet_bootstrap.cc:654] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:13.130587 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 (DEFAULT_TABLE table=default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:13.268182 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6826e4e4db2043b3a4fe9e7a9359cab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:13.278896 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_0ba9f3241f494f148bd809e66279f480" } modify_external_catalogs: true
I20260808 13:35:13.291164 6390 raft_consensus.cc:482] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:13.291926 6390 raft_consensus.cc:504] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.296317 6401 raft_consensus.cc:482] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:13.297071 6401 raft_consensus.cc:504] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.301715 6401 leader_election.cc:288] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:13.310865 6401 raft_consensus.cc:482] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:13.311446 6401 raft_consensus.cc:504] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.315552 6401 leader_election.cc:288] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:13.319768 6390 leader_election.cc:288] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:13.322676 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_fc33bfcdedd744039ea0ff6c698d01cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:13.323503 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f26e054d1992452396d3f7334ba3602c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:13.324185 5642 raft_consensus.cc:2454] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
W20260808 13:35:13.324293 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc33bfcdedd744039ea0ff6c698d01cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:13.325994 5772 leader_election.cc:302] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:35:13.256932 4835 rpcz_store.cc:269] 0808 13:35:12.476645 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:12.476951 (+ 306us) service_pool.cc:227] Handling call
0808 13:35:12.515727 (+ 38776us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:13.255509 (+739782us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":80287616}
I20260808 13:35:13.328627 6391 raft_consensus.cc:2781] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:13.329154 6391 raft_consensus.cc:482] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:13.329571 6391 raft_consensus.cc:3037] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:13.343698 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f26e054d1992452396d3f7334ba3602c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:13.346764 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d13cbde6844079c2d1eb80732bb1f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:13.347985 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 2cd2d0c9ab2c4704a74d06559bb913d6
W20260808 13:35:13.348601 5550 leader_election.cc:341] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:13.349407 6401 raft_consensus.cc:482] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:13.349990 6401 raft_consensus.cc:504] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.350795 6211 ts_tablet_manager.cc:1918] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260808 13:35:13.351176 6211 log.cc:1196] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/3e36f7753d9d4c43afc3c9b253a03846
I20260808 13:35:13.352551 6211 ts_tablet_manager.cc:1939] T 3e36f7753d9d4c43afc3c9b253a03846 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:13.353312 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:13.354558 6401 leader_election.cc:288] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:13.368111 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d388589d386408bb10ca030b87af787" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:13.371526 5550 leader_election.cc:341] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:13.374336 5845 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:13.382774 6391 raft_consensus.cc:504] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.384357 5880 raft_consensus.cc:2379] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:13.385318 6391 leader_election.cc:288] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:13.387868 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f26e054d1992452396d3f7334ba3602c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:13.388613 5642 raft_consensus.cc:3037] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:13.390738 6033 tablet_bootstrap.cc:654] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:13.408597 6131 ts_tablet_manager.cc:1918] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260808 13:35:13.409129 6131 log.cc:1196] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/3e36f7753d9d4c43afc3c9b253a03846
I20260808 13:35:13.410207 6131 ts_tablet_manager.cc:1939] T 3e36f7753d9d4c43afc3c9b253a03846 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:13.418450 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 3e36f7753d9d4c43afc3c9b253a03846 (table default.Table_3a5f4c4757564738b8e967ff71392b87 [id=e794d9029f6446a996ff766936695c20]) successfully deleted
W20260808 13:35:13.448194 4839 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 234591e698f8472e90e1f610da37ff3a has no consensus state
W20260808 13:35:13.453100 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2cd2d0c9ab2c4704a74d06559bb913d6 already in progress: creating tablet
I20260808 13:35:13.458418 6026 tablet_bootstrap.cc:492] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:13.458925 6026 ts_tablet_manager.cc:1397] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.426s user 0.016s sys 0.001s
I20260808 13:35:13.460527 6033 tablet_bootstrap.cc:492] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:13.461169 6033 ts_tablet_manager.cc:1397] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.294s user 0.014s sys 0.005s
I20260808 13:35:13.465813 6033 raft_consensus.cc:348] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.424446 6419 ts_tablet_manager.cc:1918] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260808 13:35:13.469379 6033 raft_consensus.cc:374] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:13.469688 6033 raft_consensus.cc:729] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:13.470317 6033 consensus_queue.cc:260] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.476931 6033 ts_tablet_manager.cc:1428] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
I20260808 13:35:13.477739 6033 tablet_bootstrap.cc:492] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:13.488662 6033 tablet_bootstrap.cc:654] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:13.396656 6154 raft_consensus.cc:482] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:13.492149 6154 raft_consensus.cc:504] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.497278 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 3e36f7753d9d4c43afc3c9b253a03846 (table default.Table_3a5f4c4757564738b8e967ff71392b87 [id=e794d9029f6446a996ff766936695c20]) successfully deleted
I20260808 13:35:13.494375 6154 leader_election.cc:288] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:13.510183 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f26e054d1992452396d3f7334ba3602c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:13.521484 6033 tablet_bootstrap.cc:492] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:13.521974 6033 ts_tablet_manager.cc:1397] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.044s user 0.005s sys 0.014s
I20260808 13:35:13.524957 6033 raft_consensus.cc:348] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.525624 6033 raft_consensus.cc:374] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:13.525847 6033 raft_consensus.cc:729] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:13.525997 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_02e12ed060714fadbc5b14d44b2be13c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:13.526384 6033 consensus_queue.cc:260] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:13.527873 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_02e12ed060714fadbc5b14d44b2be13c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:13.528950 6033 ts_tablet_manager.cc:1428] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260808 13:35:13.476267 6419 log.cc:1196] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/3e36f7753d9d4c43afc3c9b253a03846
I20260808 13:35:13.547044 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a95712de254c13b3b6dc4fe5cb15f7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:13.550442 6033 tablet_bootstrap.cc:492] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:13.520121 6046 tablet_bootstrap.cc:492] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:13.552954 6046 ts_tablet_manager.cc:1397] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.475s user 0.013s sys 0.001s
I20260808 13:35:13.558030 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f26e054d1992452396d3f7334ba3602c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:13.559489 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d13cbde6844079c2d1eb80732bb1f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:13.563638 6328 consensus_queue.cc:1035] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:13.567039 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f26e054d1992452396d3f7334ba3602c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:13.570749 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d388589d386408bb10ca030b87af787" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:13.571523 5771 leader_election.cc:341] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:13.573402 6033 tablet_bootstrap.cc:654] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:13.565256 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:13.579020 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_339be96b6fc64b40af37836f3d8bedc7" } modify_external_catalogs: true
I20260808 13:35:13.579578 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_339be96b6fc64b40af37836f3d8bedc7" } modify_external_catalogs: true
I20260808 13:35:13.585873 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_e37ea7df0fbb4dea827afe2023a0fac5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:13.590368 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2cd2d0c9ab2c4704a74d06559bb913d6 already in progress: creating tablet
W20260808 13:35:13.592308 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e37ea7df0fbb4dea827afe2023a0fac5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:13.599484 6419 ts_tablet_manager.cc:1939] T 3e36f7753d9d4c43afc3c9b253a03846 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:13.608875 6046 raft_consensus.cc:348] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.635327 6046 raft_consensus.cc:374] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:13.635591 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet a541ec35b1a44109866fabc953f14cc6
I20260808 13:35:13.638899 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet a541ec35b1a44109866fabc953f14cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:13.639086 6046 raft_consensus.cc:729] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:13.639482 5346 tablet_service.cc:1514] Processing DeleteTablet for tablet a541ec35b1a44109866fabc953f14cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:13.640341 6046 consensus_queue.cc:260] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:13.641211 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet a541ec35b1a44109866fabc953f14cc6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a541ec35b1a44109866fabc953f14cc6 already in progress: creating tablet
W20260808 13:35:13.641858 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet a541ec35b1a44109866fabc953f14cc6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a541ec35b1a44109866fabc953f14cc6 already in progress: creating tablet
W20260808 13:35:13.639128 6426 log.cc:924] Time spent T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: Append to log took a long time: real 0.141s user 0.000s sys 0.001s
I20260808 13:35:13.648237 6026 raft_consensus.cc:348] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.649111 6026 raft_consensus.cc:374] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:13.649444 6026 raft_consensus.cc:729] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:13.655666 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a95712de254c13b3b6dc4fe5cb15f7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:13.656477 5644 raft_consensus.cc:2379] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:13.657281 5616 tablet_service.cc:1514] Processing DeleteTablet for tablet a541ec35b1a44109866fabc953f14cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:13.659551 5642 raft_consensus.cc:2454] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
W20260808 13:35:13.650314 5551 leader_election.cc:341] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:13.664891 5551 leader_election.cc:302] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:13.666419 5551 leader_election.cc:341] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:13.668955 6428 raft_consensus.cc:2726] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:13.667057 5551 leader_election.cc:302] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:13.679579 5285 leader_election.cc:302] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:13.690085 6046 ts_tablet_manager.cc:1428] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.137s user 0.006s sys 0.001s
I20260808 13:35:13.690564 6033 tablet_bootstrap.cc:492] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:13.691030 6033 ts_tablet_manager.cc:1397] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.141s user 0.024s sys 0.029s
I20260808 13:35:13.694132 6033 raft_consensus.cc:348] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.694900 6033 raft_consensus.cc:374] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:13.695151 6033 raft_consensus.cc:729] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:13.705385 6154 raft_consensus.cc:3037] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:13.712795 6428 raft_consensus.cc:2726] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:13.713771 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 47531.5us
W20260808 13:35:13.714977 5551 leader_election.cc:341] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:13.714949 6033 consensus_queue.cc:260] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.718642 6033 ts_tablet_manager.cc:1428] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.027s user 0.004s sys 0.002s
I20260808 13:35:13.720463 6033 tablet_bootstrap.cc:492] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:13.724774 6026 consensus_queue.cc:260] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.728091 6026 ts_tablet_manager.cc:1428] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.269s user 0.003s sys 0.005s
I20260808 13:35:13.715622 5551 leader_election.cc:302] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:13.747805 6428 raft_consensus.cc:2726] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:13.750480 6457 raft_consensus.cc:482] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:13.751642 6033 tablet_bootstrap.cc:654] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:13.751448 5772 leader_election.cc:302] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:13.759716 6328 raft_consensus.cc:2781] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:13.760731 6328 raft_consensus.cc:686] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:13.761910 6328 consensus_queue.cc:237] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.755270 6046 tablet_bootstrap.cc:492] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:13.772961 6419 tablet_replica.cc:330] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:13.772955 6458 tablet.cc:1722] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:13.753954 6456 tablet.cc:1722] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260808 13:35:13.777952 6434 log.cc:924] Time spent T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Append to log took a long time: real 0.133s user 0.000s sys 0.002s
I20260808 13:35:13.784526 6428 raft_consensus.cc:482] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:13.778363 6457 raft_consensus.cc:504] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.785032 6428 raft_consensus.cc:504] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.787202 6419 raft_consensus.cc:2229] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:13.788291 6419 raft_consensus.cc:2258] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:13.793119 6033 tablet_bootstrap.cc:492] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:13.793504 6046 tablet_bootstrap.cc:654] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:13.793658 6033 ts_tablet_manager.cc:1397] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.073s user 0.010s sys 0.008s
I20260808 13:35:13.799414 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 3e36f7753d9d4c43afc3c9b253a03846 (table default.Table_3a5f4c4757564738b8e967ff71392b87 [id=e794d9029f6446a996ff766936695c20]) successfully deleted
I20260808 13:35:13.807828 6033 raft_consensus.cc:348] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.808586 6033 raft_consensus.cc:374] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:13.808856 6033 raft_consensus.cc:729] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:13.809465 6033 consensus_queue.cc:260] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.815277 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6e6ce417cc45fd90f5858c30d37a47" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:13.815886 6391 raft_consensus.cc:482] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:13.816116 5880 raft_consensus.cc:2454] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:13.816527 6391 raft_consensus.cc:504] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.817786 6457 leader_election.cc:288] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:13.819278 5550 leader_election.cc:302] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:13.819797 6391 leader_election.cc:288] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:13.820670 6401 raft_consensus.cc:2781] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:13.821103 6401 raft_consensus.cc:482] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:13.821544 6401 raft_consensus.cc:3037] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:13.829124 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "328d361c336848689c6906ca6fe2ccc3" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:13.830169 5640 raft_consensus.cc:2379] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:13.833105 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "328d361c336848689c6906ca6fe2ccc3" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:13.834170 6033 ts_tablet_manager.cc:1428] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.038s user 0.006s sys 0.000s
I20260808 13:35:13.836908 5285 leader_election.cc:302] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:13.837694 6033 tablet_bootstrap.cc:492] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:13.848240 6401 raft_consensus.cc:504] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.852213 6457 raft_consensus.cc:3037] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:13.858173 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d13cbde6844079c2d1eb80732bb1f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:13.860386 5771 leader_election.cc:341] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:13.861861 6046 tablet_bootstrap.cc:492] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:13.862376 6046 ts_tablet_manager.cc:1397] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.107s user 0.007s sys 0.012s
I20260808 13:35:13.866194 6046 raft_consensus.cc:348] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.871182 6046 raft_consensus.cc:374] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:13.871439 6046 raft_consensus.cc:729] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:13.875535 6033 tablet_bootstrap.cc:654] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:13.876298 6046 consensus_queue.cc:260] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:13.889664 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6e6ce417cc45fd90f5858c30d37a47" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:13.905434 6033 tablet_bootstrap.cc:492] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:13.905907 6033 ts_tablet_manager.cc:1397] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.068s user 0.014s sys 0.000s
I20260808 13:35:13.912591 6046 ts_tablet_manager.cc:1428] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.050s user 0.006s sys 0.005s
I20260808 13:35:13.913921 6033 raft_consensus.cc:348] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.917196 6046 tablet_bootstrap.cc:492] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:13.914580 6033 raft_consensus.cc:374] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:13.921665 6033 raft_consensus.cc:729] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:13.922345 6033 consensus_queue.cc:260] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:13.924726 4837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_909de5e2ab404a69ba774963323e2102" } modify_external_catalogs: true
I20260808 13:35:13.925395 4837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_909de5e2ab404a69ba774963323e2102" } modify_external_catalogs: true
I20260808 13:35:13.927779 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_f466df8d38e042cf9c7e3ab344f8aa49" } new_table_name: "default.Table_cba3fa83e59546fd8d7817c6ba750080" modify_external_catalogs: true
I20260808 13:35:13.933265 6033 ts_tablet_manager.cc:1428] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.027s user 0.006s sys 0.002s
I20260808 13:35:13.934392 6033 tablet_bootstrap.cc:492] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:13.934437 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_2637745cbe454854a74c76f3165627c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:13.936048 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2637745cbe454854a74c76f3165627c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:13.953136 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 60f8863bc32e49399940912444bcb972 (table default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]). Expected version 1 got 0
I20260808 13:35:13.954221 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 1 got 0
I20260808 13:35:13.958338 6046 tablet_bootstrap.cc:654] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:13.857678 6428 leader_election.cc:288] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:13.859409 6026 tablet_bootstrap.cc:492] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:13.862306 6419 ts_tablet_manager.cc:1905] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:13.895035 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d13cbde6844079c2d1eb80732bb1f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:14.009763 5640 raft_consensus.cc:2454] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:14.010511 6457 raft_consensus.cc:2726] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:14.060101 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet f0c76b819474468baac51365360becfc (table default.Table_9b3bbebdc1274b238c30a45d36cf86f7 [id=26fb789fff434bb0825daf68d5378013], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet f0c76b819474468baac51365360becfc consensus state has no leader
I20260808 13:35:14.063380 6046 tablet_bootstrap.cc:492] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:14.063846 6046 ts_tablet_manager.cc:1397] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.150s user 0.012s sys 0.004s
I20260808 13:35:14.067096 6046 raft_consensus.cc:348] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:14.067871 6046 raft_consensus.cc:374] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:14.068257 6046 raft_consensus.cc:729] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:14.069347 6046 consensus_queue.cc:260] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:14.076663 6046 ts_tablet_manager.cc:1428] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.013s user 0.008s sys 0.001s
W20260808 13:35:14.076614 6402 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.951s user 0.011s sys 0.011s
W20260808 13:35:14.077099 6402 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.951s user 0.011s sys 0.011s
I20260808 13:35:14.077491 6046 tablet_bootstrap.cc:492] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:14.087761 6046 tablet_bootstrap.cc:654] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:14.091521 6154 raft_consensus.cc:2726] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:14.094873 6456 tablet.cc:1722] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:14.100268 6458 tablet.cc:1722] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:13.955384 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]). Expected version 1 got 0
I20260808 13:35:14.112389 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (table default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]). Expected version 1 got 0
I20260808 13:35:14.129786 6033 tablet_bootstrap.cc:654] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:14.130306 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6e6ce417cc45fd90f5858c30d37a47" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:14.135177 5551 leader_election.cc:341] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:14.136529 5551 leader_election.cc:341] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:14.145467 6026 tablet_bootstrap.cc:654] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:14.111508 6401 leader_election.cc:288] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:14.176208 6485 raft_consensus.cc:482] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:14.176767 6485 raft_consensus.cc:504] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.187744 6487 tablet.cc:1722] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:14.198887 5378 raft_consensus.cc:3037] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.120990 6472 tablet.cc:1722] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:14.204759 6485 leader_election.cc:288] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:14.212625 6046 tablet_bootstrap.cc:492] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
W20260808 13:35:14.191779 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 95470.7us
I20260808 13:35:14.195250 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6e6ce417cc45fd90f5858c30d37a47" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:14.216604 6477 consensus_queue.cc:1035] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:14.218966 5880 raft_consensus.cc:3037] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.215392 6458 tablet.cc:1722] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:14.220510 6478 consensus_queue.cc:1035] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:14.224066 5772 leader_election.cc:302] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:14.227824 6467 consensus_queue.cc:1035] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260808 13:35:14.241189 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc0c6ab755c4a6887dabeeef9364201" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:14.219611 6046 ts_tablet_manager.cc:1397] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.142s user 0.014s sys 0.006s
I20260808 13:35:14.247478 6046 raft_consensus.cc:348] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.248183 6046 raft_consensus.cc:374] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:14.248469 6046 raft_consensus.cc:729] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:14.267573 6419 ts_tablet_manager.cc:1918] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:14.268512 5880 raft_consensus.cc:2454] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:14.267949 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc0c6ab755c4a6887dabeeef9364201" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:14.271478 6419 log.cc:1196] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/78d492406c8a4c4186131e02a1559cb4
I20260808 13:35:14.227267 6479 raft_consensus.cc:2781] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:14.272583 6479 raft_consensus.cc:482] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:14.273151 6479 raft_consensus.cc:3037] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:14.273334 5771 leader_election.cc:341] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:14.274077 6026 tablet_bootstrap.cc:492] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:14.274678 6026 ts_tablet_manager.cc:1397] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.416s user 0.009s sys 0.009s
I20260808 13:35:14.224045 6484 consensus_queue.cc:1035] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260808 13:35:14.278481 6026 raft_consensus.cc:348] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.279304 6026 raft_consensus.cc:374] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:14.279611 6026 raft_consensus.cc:729] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:14.280354 6026 consensus_queue.cc:260] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:14.283465 4832 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9260370cf8cb46a7b699581e4e8d46db consensus state has no leader
I20260808 13:35:14.283560 6419 ts_tablet_manager.cc:1939] T 78d492406c8a4c4186131e02a1559cb4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:14.284265 4832 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 148) took 1104 ms. Trace:
I20260808 13:35:14.287426 6026 ts_tablet_manager.cc:1428] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260808 13:35:14.289289 4838 master_service.cc:946] ReplaceTablet: received request to replace tablet f0c76b819474468baac51365360becfc from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:14.284641 4832 rpcz_store.cc:276] 0808 13:35:13.179643 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:13.179787 (+ 144us) service_pool.cc:227] Handling call
0808 13:35:13.281872 (+102085us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:13.283242 (+ 1370us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:13.284535 (+ 1293us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:13.284880 (+ 345us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:14.282559 (+997679us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:14.284192 (+ 1633us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:14.125513 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:14.125911 (+ 398us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:14.125935 (+ 24us) write_op.cc:435] Acquired schema lock
0808 13:35:14.125946 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:14.126228 (+ 282us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:14.126267 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:14.126289 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:14.126431 (+ 142us) tablet.cc:699] Row locks acquired
0808 13:35:14.126444 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:14.126555 (+ 111us) write_op.cc:270] Start()
0808 13:35:14.126693 (+ 138us) write_op.cc:276] Timestamp: P: 1786196114126465 usec, L: 0
0808 13:35:14.126712 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:14.153854 (+ 27142us) log.cc:842] Serialized 405 byte log entry
0808 13:35:14.277786 (+123932us) op_driver.cc:444] REPLICATION: finished
0808 13:35:14.280407 (+ 2621us) write_op.cc:301] APPLY: starting
0808 13:35:14.280498 (+ 91us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:14.280644 (+ 146us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:14.280817 (+ 173us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:14.281012 (+ 195us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:14.281230 (+ 218us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:14.281284 (+ 54us) write_op.cc:312] APPLY: finished
0808 13:35:14.281476 (+ 192us) log.cc:842] Serialized 23 byte log entry
0808 13:35:14.281645 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:14.281732 (+ 87us) write_op.cc:454] Released schema lock
0808 13:35:14.281767 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:14.281977 (+ 210us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:14.282019 (+ 42us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2336,"apply.run_cpu_time_us":1849,"apply.run_wall_time_us":1848,"mutex_wait_us":48037,"num_ops":1,"prepare.queue_time_us":794410,"prepare.run_cpu_time_us":2268,"prepare.run_wall_time_us":37914,"raft.queue_time_us":81,"raft.run_cpu_time_us":935,"raft.run_wall_time_us":1239,"replication_time_us":150961}]]}
I20260808 13:35:14.288183 6046 consensus_queue.cc:260] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.297259 4832 master_service.cc:946] ReplaceTablet: received request to replace tablet 2173e97590be43f9a24f55689d796269 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:14.288336 6026 tablet_bootstrap.cc:492] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:14.309918 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_280b5d6fc90b4748b675e41e2e7071f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:14.310964 6419 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 95) took 1050 ms. Trace:
W20260808 13:35:14.312103 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_280b5d6fc90b4748b675e41e2e7071f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:14.311652 6419 rpcz_store.cc:276] 0808 13:35:13.261098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:13.261328 (+ 230us) service_pool.cc:227] Handling call
0808 13:35:13.772720 (+511392us) ts_tablet_manager.cc:1145] Deleting tablet 78d492406c8a4c4186131e02a1559cb4
0808 13:35:13.772774 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:14.207087 (+434313us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:14.267232 (+ 60145us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:14.310859 (+ 43627us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":510295}
I20260808 13:35:14.318300 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 78d492406c8a4c4186131e02a1559cb4 (table default.Table_6cb245b3d74d4347843035902b8d49d1 [id=c0806126263c41b68137cad5e21de125]) successfully deleted
W20260808 13:35:14.322502 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2cd2d0c9ab2c4704a74d06559bb913d6 already in progress: creating tablet
W20260808 13:35:14.323243 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet a541ec35b1a44109866fabc953f14cc6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a541ec35b1a44109866fabc953f14cc6 already in progress: creating tablet
I20260808 13:35:14.325425 4840 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet c6332bdf63a2403699eb74ba1c636dc3
I20260808 13:35:14.333045 6026 tablet_bootstrap.cc:654] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:14.333586 6033 tablet_bootstrap.cc:492] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:14.334055 6033 ts_tablet_manager.cc:1397] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.400s user 0.009s sys 0.009s
I20260808 13:35:14.342653 6046 ts_tablet_manager.cc:1428] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.098s user 0.005s sys 0.001s
I20260808 13:35:14.342860 5550 leader_election.cc:302] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:14.343812 6046 tablet_bootstrap.cc:492] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:14.344194 6477 raft_consensus.cc:2781] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:14.344703 6477 raft_consensus.cc:686] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:14.345734 6477 consensus_queue.cc:237] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:14.352770 4823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111) [suppressed 4 similar messages]
W20260808 13:35:14.374385 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 96342.8us
I20260808 13:35:14.375304 6467 raft_consensus.cc:482] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:14.374150 6456 tablet.cc:1722] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:14.376075 6467 raft_consensus.cc:504] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:14.380954 5772 leader_election.cc:341] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:14.380895 6485 consensus_queue.cc:1035] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:14.381608 5772 leader_election.cc:302] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:14.393050 6477 consensus_queue.cc:1035] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:14.408955 6467 leader_election.cc:288] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:14.426322 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 973a7afdcbf944aab8f11b3fd3995865 consensus state has no leader
I20260808 13:35:14.426819 6478 raft_consensus.cc:2726] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:14.427676 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 46758.5us
I20260808 13:35:14.429066 6033 raft_consensus.cc:348] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.429742 6485 raft_consensus.cc:482] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:14.430225 6485 raft_consensus.cc:504] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.430837 6467 consensus_queue.cc:1035] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:14.432148 6485 leader_election.cc:288] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:14.426035 6154 raft_consensus.cc:482] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:14.438344 6479 raft_consensus.cc:504] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.439811 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc9a6ec9a86431fb46e50fcaa658d33" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:14.439850 5340 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 89) took 1240 ms. Trace:
I20260808 13:35:14.440376 5340 rpcz_store.cc:276] 0808 13:35:13.199745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:13.199901 (+ 156us) service_pool.cc:227] Handling call
0808 13:35:13.201904 (+ 2003us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:13.202022 (+ 118us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:14.003265 (+801243us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:14.439316 (+436051us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:14.439790 (+ 474us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:14.441268 6046 tablet_bootstrap.cc:654] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:14.429860 6033 raft_consensus.cc:374] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:14.442389 6033 raft_consensus.cc:729] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:14.443071 6033 consensus_queue.cc:260] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.444653 6033 ts_tablet_manager.cc:1428] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.110s user 0.005s sys 0.000s
I20260808 13:35:14.445474 6033 tablet_bootstrap.cc:492] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:14.445370 5771 leader_election.cc:341] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:14.442307 6154 raft_consensus.cc:504] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.443955 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d985de8de040829569d1f5783c6fef" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:14.447338 6154 leader_election.cc:288] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:14.448589 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa77744bcfb4a5cace730fd9f9fd1ca" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:14.449313 5642 raft_consensus.cc:2454] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:14.450670 5285 leader_election.cc:302] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:14.451696 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d13cbde6844079c2d1eb80732bb1f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:14.453254 5771 leader_election.cc:341] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:14.453855 5771 leader_election.cc:341] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:14.454499 6154 raft_consensus.cc:2781] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:14.454847 6154 raft_consensus.cc:482] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:14.455149 6154 raft_consensus.cc:3037] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.457144 6479 leader_election.cc:288] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:14.459417 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa77744bcfb4a5cace730fd9f9fd1ca" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:14.461079 6046 tablet_bootstrap.cc:492] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:14.461550 6046 ts_tablet_manager.cc:1397] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.118s user 0.010s sys 0.003s
I20260808 13:35:14.461210 5882 raft_consensus.cc:2454] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:14.463856 6046 raft_consensus.cc:348] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.464484 6046 raft_consensus.cc:374] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:14.464740 6046 raft_consensus.cc:729] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:14.465489 6046 consensus_queue.cc:260] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.467555 6046 ts_tablet_manager.cc:1428] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 13:35:14.462122 6033 tablet_bootstrap.cc:654] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:14.470865 6487 tablet.cc:1722] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:14.472409 6046 tablet_bootstrap.cc:492] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:14.478062 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d985de8de040829569d1f5783c6fef" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:14.478690 5640 raft_consensus.cc:2454] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:14.493505 6046 tablet_bootstrap.cc:654] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:14.499812 6154 raft_consensus.cc:504] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.511126 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_60e5f930145b4e7db14190e5896be3f4" } new_table_name: "default.Table_0a3697cdad3740a1baa47b3075b5e1f9" modify_external_catalogs: true
I20260808 13:35:14.512390 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_7bf999991a5c4f998994907960315c0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:14.514566 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7bf999991a5c4f998994907960315c0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:14.518381 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa77744bcfb4a5cace730fd9f9fd1ca" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:14.519516 5638 raft_consensus.cc:3037] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.534169 4835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_4a3f8ae1263244bd80f5eb36424e85e7" } modify_external_catalogs: true
I20260808 13:35:14.534847 4835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_4a3f8ae1263244bd80f5eb36424e85e7" } modify_external_catalogs: true
I20260808 13:35:14.538527 6046 tablet_bootstrap.cc:492] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:14.538957 6046 ts_tablet_manager.cc:1397] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.067s user 0.007s sys 0.005s
I20260808 13:35:14.540226 6154 leader_election.cc:288] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:14.541476 5638 raft_consensus.cc:2454] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:14.541666 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa77744bcfb4a5cace730fd9f9fd1ca" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:14.547621 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc9a6ec9a86431fb46e50fcaa658d33" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:14.567376 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d13cbde6844079c2d1eb80732bb1f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:14.568374 5637 raft_consensus.cc:3037] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.571856 5383 raft_consensus.cc:3037] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.581904 6026 tablet_bootstrap.cc:492] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:14.582379 6026 ts_tablet_manager.cc:1397] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.294s user 0.011s sys 0.025s
I20260808 13:35:14.542492 6046 raft_consensus.cc:348] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.583829 6046 raft_consensus.cc:374] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:14.584177 6046 raft_consensus.cc:729] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:14.583750 5285 leader_election.cc:302] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:14.585134 6154 raft_consensus.cc:2781] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:14.589181 6046 consensus_queue.cc:260] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.593675 6026 raft_consensus.cc:348] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.594383 6026 raft_consensus.cc:374] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:14.596058 6026 raft_consensus.cc:729] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:14.596684 6026 consensus_queue.cc:260] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.598053 6046 ts_tablet_manager.cc:1428] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.059s user 0.007s sys 0.000s
I20260808 13:35:14.598081 6154 raft_consensus.cc:686] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:14.600296 6154 consensus_queue.cc:237] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.602049 6046 tablet_bootstrap.cc:492] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:14.618948 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet f0c76b819474468baac51365360becfc
I20260808 13:35:14.620479 4838 catalog_manager.cc:6565] ReplaceTablet: tablet f0c76b819474468baac51365360becfc deleted and replaced by tablet 4b7284afe0294644a368cf51451c7ab6
I20260808 13:35:14.622800 6457 raft_consensus.cc:482] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:14.623528 6457 raft_consensus.cc:504] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.614382 5637 raft_consensus.cc:2454] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:14.626804 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:14.634280 6457 leader_election.cc:288] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:14.634492 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:14.638916 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet f0c76b819474468baac51365360becfc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f0c76b819474468baac51365360becfc already in progress: creating tablet
I20260808 13:35:14.649384 6026 ts_tablet_manager.cc:1428] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.067s user 0.002s sys 0.005s
I20260808 13:35:14.650573 6026 tablet_bootstrap.cc:492] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:14.642731 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet f0c76b819474468baac51365360becfc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f0c76b819474468baac51365360becfc already in progress: creating tablet
I20260808 13:35:14.644944 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc0c6ab755c4a6887dabeeef9364201" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:14.660820 5285 leader_election.cc:341] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:14.669034 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a8fa4ae765254ab895c787d8fcc8f5ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:14.671093 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8fa4ae765254ab895c787d8fcc8f5ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:14.542405 5882 raft_consensus.cc:3037] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.695552 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:14.697265 6457 raft_consensus.cc:482] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:14.697923 6457 raft_consensus.cc:504] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.702510 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d985de8de040829569d1f5783c6fef" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:14.703393 5637 raft_consensus.cc:2454] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:14.702908 6026 tablet_bootstrap.cc:654] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:14.710636 5285 leader_election.cc:302] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:14.715483 6457 leader_election.cc:288] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:14.717352 6457 raft_consensus.cc:2781] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:14.696019 6328 consensus_queue.cc:1035] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260808 13:35:14.721026 6457 raft_consensus.cc:482] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:14.722134 6457 raft_consensus.cc:3037] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.699756 5772 leader_election.cc:302] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:14.727178 5772 leader_election.cc:341] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:14.727793 5772 leader_election.cc:302] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:14.729804 6467 raft_consensus.cc:2781] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:14.730324 6467 raft_consensus.cc:482] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:14.730927 6467 raft_consensus.cc:3037] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.736317 6478 raft_consensus.cc:2726] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:14.737282 6478 consensus_queue.cc:1035] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:14.740688 6026 tablet_bootstrap.cc:492] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:14.741185 6026 ts_tablet_manager.cc:1397] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.091s user 0.006s sys 0.018s
I20260808 13:35:14.746279 6026 raft_consensus.cc:348] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.754483 6026 raft_consensus.cc:374] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:14.769197 6046 tablet_bootstrap.cc:654] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:14.754967 6026 raft_consensus.cc:729] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:14.773470 6026 consensus_queue.cc:260] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.788702 6026 ts_tablet_manager.cc:1428] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.047s user 0.008s sys 0.000s
W20260808 13:35:14.778855 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 72141us
I20260808 13:35:14.797166 6026 tablet_bootstrap.cc:492] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:14.736773 5772 leader_election.cc:302] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:14.822736 6026 tablet_bootstrap.cc:654] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:14.821897 6485 raft_consensus.cc:2781] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:14.827553 6485 raft_consensus.cc:686] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:14.828397 6485 consensus_queue.cc:237] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:14.823320 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 86724.7us
I20260808 13:35:14.843312 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc0c6ab755c4a6887dabeeef9364201" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:14.844309 5880 raft_consensus.cc:2454] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:14.861364 4838 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6008751431314433bd054456abced5f0" } new_table_name: "default.Table_d30a1f7564df4e6085a1858c6f3924fc" modify_external_catalogs: true
I20260808 13:35:14.864418 5283 leader_election.cc:302] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:14.866344 6539 raft_consensus.cc:2781] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:14.866838 6539 raft_consensus.cc:482] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:14.867393 6539 raft_consensus.cc:3037] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.876063 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_13f833fb48504c76865676e23461e6e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:14.877794 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13f833fb48504c76865676e23461e6e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:14.890450 6457 raft_consensus.cc:504] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.915664 6467 raft_consensus.cc:504] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.921697 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d985de8de040829569d1f5783c6fef" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:14.922593 5640 raft_consensus.cc:3037] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.920351 6457 leader_election.cc:288] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:14.928643 6033 tablet_bootstrap.cc:492] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:14.929124 6033 ts_tablet_manager.cc:1397] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.484s user 0.016s sys 0.006s
I20260808 13:35:14.932256 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d985de8de040829569d1f5783c6fef" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:14.916965 5882 raft_consensus.cc:2454] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:14.954967 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d985de8de040829569d1f5783c6fef" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:14.959797 5883 raft_consensus.cc:2379] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:14.960146 6467 leader_election.cc:288] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:14.964203 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d985de8de040829569d1f5783c6fef" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:14.965389 5378 raft_consensus.cc:2379] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:14.965426 6033 raft_consensus.cc:348] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.966142 6033 raft_consensus.cc:374] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:14.966418 6033 raft_consensus.cc:729] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:14.967074 6033 consensus_queue.cc:260] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.941908 6539 raft_consensus.cc:504] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:14.978535 6046 tablet_bootstrap.cc:492] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:14.981365 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d985de8de040829569d1f5783c6fef" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:14.983578 6033 ts_tablet_manager.cc:1428] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.054s user 0.004s sys 0.003s
I20260808 13:35:14.984936 6033 tablet_bootstrap.cc:492] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:14.984470 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc0c6ab755c4a6887dabeeef9364201" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:14.985357 5638 raft_consensus.cc:3037] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.983114 6539 leader_election.cc:288] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:14.987133 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc0c6ab755c4a6887dabeeef9364201" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:14.987702 5883 raft_consensus.cc:3037] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:14.991652 5772 leader_election.cc:302] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:14.993001 6467 raft_consensus.cc:2726] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 13:35:15.016548 5883 raft_consensus.cc:2454] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:15.023445 5283 leader_election.cc:302] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:15.024456 6539 raft_consensus.cc:2781] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:15.025547 6456 tablet.cc:1722] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:15.029687 6557 tablet.cc:1722] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:15.039701 6539 raft_consensus.cc:686] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:15.040681 6539 consensus_queue.cc:237] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.047801 5640 raft_consensus.cc:2454] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:15.048916 5285 leader_election.cc:302] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:15.049997 6457 raft_consensus.cc:2781] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:15.050427 6457 raft_consensus.cc:686] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:15.051158 6457 consensus_queue.cc:237] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.058816 6033 tablet_bootstrap.cc:654] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:15.066278 6472 tablet.cc:1722] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260808 13:35:15.078295 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 973a7afdcbf944aab8f11b3fd3995865 (table default.Table_cba3fa83e59546fd8d7817c6ba750080 [id=eb20921cee2f48dc86300ce8ce5f52a1], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 973a7afdcbf944aab8f11b3fd3995865 consensus state has no leader
I20260808 13:35:15.083751 6026 tablet_bootstrap.cc:492] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:15.084474 6026 ts_tablet_manager.cc:1397] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.289s user 0.017s sys 0.008s
I20260808 13:35:15.086696 6026 raft_consensus.cc:348] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.087599 6026 raft_consensus.cc:374] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:15.087837 6026 raft_consensus.cc:729] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
W20260808 13:35:15.097092 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet f0c76b819474468baac51365360becfc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f0c76b819474468baac51365360becfc already in progress: creating tablet
I20260808 13:35:15.098351 6401 raft_consensus.cc:482] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:15.099520 6401 raft_consensus.cc:504] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.101402 6401 leader_election.cc:288] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:15.102128 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc9a6ec9a86431fb46e50fcaa658d33" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:15.103796 5551 leader_election.cc:341] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:15.120404 6033 tablet_bootstrap.cc:492] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:15.120512 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc9a6ec9a86431fb46e50fcaa658d33" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:15.120818 6033 ts_tablet_manager.cc:1397] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.136s user 0.012s sys 0.003s
I20260808 13:35:15.121099 5883 raft_consensus.cc:2454] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:15.121788 6046 ts_tablet_manager.cc:1397] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.520s user 0.007s sys 0.016s
I20260808 13:35:15.124925 6033 raft_consensus.cc:348] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.125772 6033 raft_consensus.cc:374] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:15.126078 6033 raft_consensus.cc:729] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:15.123327 6026 consensus_queue.cc:260] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.123317 5550 leader_election.cc:302] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:15.128404 6401 raft_consensus.cc:2781] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:15.129498 6401 raft_consensus.cc:482] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:15.130105 6401 raft_consensus.cc:3037] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:15.142464 6033 consensus_queue.cc:260] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.145210 6033 ts_tablet_manager.cc:1428] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.024s user 0.004s sys 0.004s
I20260808 13:35:15.146255 6033 tablet_bootstrap.cc:492] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:15.158912 6026 ts_tablet_manager.cc:1428] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.074s user 0.006s sys 0.000s
I20260808 13:35:15.160565 5638 raft_consensus.cc:2454] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:15.174304 6033 tablet_bootstrap.cc:654] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:15.175496 6026 tablet_bootstrap.cc:492] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:15.169354 6401 raft_consensus.cc:504] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.185084 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc9a6ec9a86431fb46e50fcaa658d33" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:15.186911 5551 leader_election.cc:341] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:15.217036 6046 raft_consensus.cc:348] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:15.217785 6046 raft_consensus.cc:374] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:15.218052 6046 raft_consensus.cc:729] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:15.223567 6401 leader_election.cc:288] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:15.226898 4835 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 85d931c8944342b6938a3d21a242f466
I20260808 13:35:15.226974 6046 consensus_queue.cc:260] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:15.238217 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc9a6ec9a86431fb46e50fcaa658d33" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:15.238873 5883 raft_consensus.cc:3037] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:15.246551 5883 raft_consensus.cc:2454] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:15.246459 6046 ts_tablet_manager.cc:1428] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.124s user 0.003s sys 0.005s
I20260808 13:35:15.251768 5550 leader_election.cc:302] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:15.254320 6402 raft_consensus.cc:2781] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:15.254791 6402 raft_consensus.cc:686] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:15.255523 6402 consensus_queue.cc:237] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.257194 6046 tablet_bootstrap.cc:492] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:15.269115 4838 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f66b38e83dab4bf1a3d2d1e088a96988 has no consensus state
I20260808 13:35:15.270141 4838 master_service.cc:946] ReplaceTablet: received request to replace tablet 543debff6da54d26bf62260bf8721ff7 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:15.278082 6026 tablet_bootstrap.cc:654] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:15.283473 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_1fe1ec319d834d319cd9c35e801b1c91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:15.285604 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fe1ec319d834d319cd9c35e801b1c91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:15.295270 4838 master_service.cc:946] ReplaceTablet: received request to replace tablet 8948ce98e7a941f59fa9932fe241638d from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:15.314553 6046 tablet_bootstrap.cc:654] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:15.333854 6026 tablet_bootstrap.cc:492] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:15.334856 6026 ts_tablet_manager.cc:1397] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.160s user 0.012s sys 0.000s
I20260808 13:35:15.340456 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 1 got 0
I20260808 13:35:15.341478 4839 catalog_manager.cc:5495] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:15.342933 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (table default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]). Expected version 1 got 0
I20260808 13:35:15.342962 5835 tablet_service.cc:1467] Processing CreateTablet for tablet e00f2e0b98af431d99894e8778257bc4 (DEFAULT_TABLE table=default.Table_8523d4a015194c64b22f66a22b0dcc06 [id=b285a956530e4fb4a5d1632a45d87c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.344147 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e00f2e0b98af431d99894e8778257bc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.343842 4839 catalog_manager.cc:5495] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:15.345788 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]). Expected version 1 got 0
I20260808 13:35:15.346513 4839 catalog_manager.cc:5495] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:15.347698 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
I20260808 13:35:15.348701 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 60f8863bc32e49399940912444bcb972 (table default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]). Expected version 1 got 0
I20260808 13:35:15.348773 5340 tablet_service.cc:1467] Processing CreateTablet for tablet e00f2e0b98af431d99894e8778257bc4 (DEFAULT_TABLE table=default.Table_8523d4a015194c64b22f66a22b0dcc06 [id=b285a956530e4fb4a5d1632a45d87c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.349601 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]). Expected version 1 got 0
I20260808 13:35:15.349989 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e00f2e0b98af431d99894e8778257bc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.362864 6026 raft_consensus.cc:348] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.363718 6026 raft_consensus.cc:374] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:15.364005 6026 raft_consensus.cc:729] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:15.363857 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 127e8d0323954ef788018bcb91582951 (DEFAULT_TABLE table=default.Table_23396b27c08444c7981e55a66139f6b6 [id=2004b79d44ab440ba8d0004db91b6bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.365803 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 127e8d0323954ef788018bcb91582951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.368294 6328 raft_consensus.cc:482] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:15.369026 6328 raft_consensus.cc:504] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.371208 5840 tablet_service.cc:1467] Processing CreateTablet for tablet 31fc9fdb5ef543da833f9af6ca5807a9 (DEFAULT_TABLE table=default.Table_faaabef2a0ab42648427c33cda679fb7 [id=7afc114b9ab641df8c540f834036b70c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.372505 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fc9fdb5ef543da833f9af6ca5807a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.376526 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b809b185ee9e4c658a050c41c9abc4ad" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:15.374110 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b809b185ee9e4c658a050c41c9abc4ad" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:15.379273 5771 leader_election.cc:341] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:15.380208 6328 leader_election.cc:288] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:15.380625 5608 tablet_service.cc:1467] Processing CreateTablet for tablet e00f2e0b98af431d99894e8778257bc4 (DEFAULT_TABLE table=default.Table_8523d4a015194c64b22f66a22b0dcc06 [id=b285a956530e4fb4a5d1632a45d87c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.382195 6046 tablet_bootstrap.cc:492] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:15.382027 6428 consensus_queue.cc:1035] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:15.382943 6046 ts_tablet_manager.cc:1397] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.126s user 0.010s sys 0.004s
I20260808 13:35:15.386921 6046 raft_consensus.cc:348] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.387733 6046 raft_consensus.cc:374] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:15.388043 6046 raft_consensus.cc:729] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:15.387754 5609 tablet_service.cc:1467] Processing CreateTablet for tablet 127e8d0323954ef788018bcb91582951 (DEFAULT_TABLE table=default.Table_23396b27c08444c7981e55a66139f6b6 [id=2004b79d44ab440ba8d0004db91b6bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.388746 6046 consensus_queue.cc:260] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.392457 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e00f2e0b98af431d99894e8778257bc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.393333 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 127e8d0323954ef788018bcb91582951 (DEFAULT_TABLE table=default.Table_23396b27c08444c7981e55a66139f6b6 [id=2004b79d44ab440ba8d0004db91b6bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.395697 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 127e8d0323954ef788018bcb91582951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.397905 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 127e8d0323954ef788018bcb91582951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.370898 6026 consensus_queue.cc:260] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.405588 5638 raft_consensus.cc:2454] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:15.410163 6026 ts_tablet_manager.cc:1428] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.075s user 0.005s sys 0.003s
I20260808 13:35:15.409703 5772 leader_election.cc:302] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:15.411201 6026 tablet_bootstrap.cc:492] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:15.415180 6328 raft_consensus.cc:2781] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:15.415634 6328 raft_consensus.cc:482] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:15.416083 6328 raft_consensus.cc:3037] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:15.426512 6328 raft_consensus.cc:504] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.429159 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b809b185ee9e4c658a050c41c9abc4ad" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:15.430755 5771 leader_election.cc:341] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:15.434998 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 31fc9fdb5ef543da833f9af6ca5807a9 (DEFAULT_TABLE table=default.Table_faaabef2a0ab42648427c33cda679fb7 [id=7afc114b9ab641df8c540f834036b70c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.436127 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fc9fdb5ef543da833f9af6ca5807a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.437047 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 31fc9fdb5ef543da833f9af6ca5807a9 (DEFAULT_TABLE table=default.Table_faaabef2a0ab42648427c33cda679fb7 [id=7afc114b9ab641df8c540f834036b70c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.438552 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fc9fdb5ef543da833f9af6ca5807a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.440122 6328 leader_election.cc:288] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:15.441676 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b809b185ee9e4c658a050c41c9abc4ad" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:15.442341 5638 raft_consensus.cc:3037] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:15.452064 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 234591e698f8472e90e1f610da37ff3a (DEFAULT_TABLE table=default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.452375 5840 tablet_service.cc:1467] Processing CreateTablet for tablet 234591e698f8472e90e1f610da37ff3a (DEFAULT_TABLE table=default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.453251 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 234591e698f8472e90e1f610da37ff3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.453418 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 234591e698f8472e90e1f610da37ff3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.458163 6428 raft_consensus.cc:482] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:15.458630 6428 raft_consensus.cc:504] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.461941 6428 leader_election.cc:288] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:15.462728 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29786cb404bc2b01473c3b8390a50" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:15.462935 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29786cb404bc2b01473c3b8390a50" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:15.463953 5550 leader_election.cc:341] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:15.465162 5551 leader_election.cc:341] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:15.465902 5551 leader_election.cc:302] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:15.466732 6428 raft_consensus.cc:2726] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:15.467262 5638 raft_consensus.cc:2454] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:15.468338 5772 leader_election.cc:302] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:15.469264 6328 raft_consensus.cc:2781] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:15.473668 6328 raft_consensus.cc:686] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:15.480357 5607 tablet_service.cc:1467] Processing CreateTablet for tablet 234591e698f8472e90e1f610da37ff3a (DEFAULT_TABLE table=default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.478825 6328 consensus_queue.cc:237] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.481889 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 234591e698f8472e90e1f610da37ff3a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:15.500061 5551 consensus_peers.cc:487] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:15.502899 6457 consensus_queue.cc:1035] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:15.505686 5840 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f (DEFAULT_TABLE table=default.Table_b75082dac94348fab668160895573637 [id=56f4695998f34301b8528a79b16e2dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.506946 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd6945ecccc4684a3fa5b2a1f07057f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.514866 6046 ts_tablet_manager.cc:1428] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.132s user 0.007s sys 0.001s
I20260808 13:35:15.535727 6457 consensus_queue.cc:1035] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:15.547225 6026 tablet_bootstrap.cc:654] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:15.554160 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f (DEFAULT_TABLE table=default.Table_b75082dac94348fab668160895573637 [id=56f4695998f34301b8528a79b16e2dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.556025 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd6945ecccc4684a3fa5b2a1f07057f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.558552 6046 tablet_bootstrap.cc:492] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:15.588377 5357 tablet_service.cc:1467] Processing CreateTablet for tablet fcf907a9f19f4d64b27be946011a8ddb (DEFAULT_TABLE table=default.Table_912bf698cf184d5690eeba3080b2b9bf [id=d99ab852cfb94fb4835c8e9c19d293fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.589732 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf907a9f19f4d64b27be946011a8ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.594790 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 8948ce98e7a941f59fa9932fe241638d
I20260808 13:35:15.597185 4838 catalog_manager.cc:6565] ReplaceTablet: tablet 8948ce98e7a941f59fa9932fe241638d deleted and replaced by tablet d9268f5f6dee4112932e106010a3c2be
I20260808 13:35:15.599649 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 8948ce98e7a941f59fa9932fe241638d with delete_type TABLET_DATA_DELETED (replaced by tablet d9268f5f6dee4112932e106010a3c2be) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:15.606972 4837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 45a26598a69146e1950e9227523fff13
I20260808 13:35:15.607175 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet 8948ce98e7a941f59fa9932fe241638d with delete_type TABLET_DATA_DELETED (replaced by tablet d9268f5f6dee4112932e106010a3c2be) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:15.608760 4837 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 157) took 1759 ms. Trace:
I20260808 13:35:15.611706 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a26598a69146e1950e9227523fff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:15.612087 5346 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a26598a69146e1950e9227523fff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:15.609098 4837 rpcz_store.cc:276] 0808 13:35:13.849758 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:13.849913 (+ 155us) service_pool.cc:227] Handling call
0808 13:35:13.924584 (+ 74671us) spinlock_profiling.cc:243] Waited 73.6 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff466 00007f227bd40b34 00007f227bd40ac2 00007f227bd409f1 00007f227bd4096c 00007f227bd3fc22 00007f2279fde94c 00007f2279fddd86
0808 13:35:13.925930 (+ 1346us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:13.933129 (+ 7199us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:13.933350 (+ 221us) catalog_manager.cc:2761] Locking tablets
0808 13:35:15.339282 (+1405932us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:15.606536 (+267254us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:15.606768 (+ 232us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:15.606882 (+ 114us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:15.608689 (+ 1807us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:15.421307 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:15.421666 (+ 359us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:15.421687 (+ 21us) write_op.cc:435] Acquired schema lock
0808 13:35:15.421699 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:15.422026 (+ 327us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:15.422061 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:15.422083 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:15.422364 (+ 281us) tablet.cc:699] Row locks acquired
0808 13:35:15.422378 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:15.422493 (+ 115us) write_op.cc:270] Start()
0808 13:35:15.422574 (+ 81us) write_op.cc:276] Timestamp: P: 1786196115422432 usec, L: 0
0808 13:35:15.422592 (+ 18us) op_driver.cc:331] REPLICATION: starting
0808 13:35:15.548885 (+126293us) log.cc:842] Serialized 725 byte log entry
0808 13:35:15.550044 (+ 1159us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:15.599372 (+ 49328us) op_driver.cc:444] REPLICATION: finished
0808 13:35:15.605141 (+ 5769us) write_op.cc:301] APPLY: starting
0808 13:35:15.605191 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:15.605301 (+ 110us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:15.605322 (+ 21us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:15.605546 (+ 224us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:15.605677 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:15.605717 (+ 40us) write_op.cc:312] APPLY: finished
0808 13:35:15.605896 (+ 179us) log.cc:842] Serialized 29 byte log entry
0808 13:35:15.606227 (+ 331us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:15.606303 (+ 76us) write_op.cc:454] Released schema lock
0808 13:35:15.606334 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":169268608,"child_traces":[["op",{"apply.queue_time_us":5606,"num_ops":2,"prepare.queue_time_us":80067,"prepare.run_cpu_time_us":4126,"prepare.run_wall_time_us":145677,"raft.queue_time_us":118,"raft.run_cpu_time_us":981,"raft.run_wall_time_us":982,"replication_time_us":176700,"spinlock_wait_cycles":289191808}]]}
I20260808 13:35:15.616333 4841 catalog_manager.cc:5495] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:15.617846 4841 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]). Expected version 1 got 0
W20260808 13:35:15.621451 4823 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:37135, user_credentials={real_user=slave}} blocked reactor thread for 97843.9us
I20260808 13:35:15.632298 6033 tablet_bootstrap.cc:492] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:15.632761 6033 ts_tablet_manager.cc:1397] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.487s user 0.058s sys 0.072s
I20260808 13:35:15.636313 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f (DEFAULT_TABLE table=default.Table_b75082dac94348fab668160895573637 [id=56f4695998f34301b8528a79b16e2dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.638113 6585 tablet_replica.cc:330] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:15.638118 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 8948ce98e7a941f59fa9932fe241638d with delete_type TABLET_DATA_DELETED (replaced by tablet d9268f5f6dee4112932e106010a3c2be) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:15.638942 5616 tablet_service.cc:1467] Processing CreateTablet for tablet fcf907a9f19f4d64b27be946011a8ddb (DEFAULT_TABLE table=default.Table_912bf698cf184d5690eeba3080b2b9bf [id=d99ab852cfb94fb4835c8e9c19d293fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.638836 6033 raft_consensus.cc:348] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.639513 6033 raft_consensus.cc:374] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:15.639707 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a26598a69146e1950e9227523fff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:15.640973 6587 tablet_replica.cc:330] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:15.641577 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd6945ecccc4684a3fa5b2a1f07057f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.643317 6585 raft_consensus.cc:2229] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:15.647171 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf907a9f19f4d64b27be946011a8ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.647701 6046 tablet_bootstrap.cc:654] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:15.647852 6585 raft_consensus.cc:2258] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:15.642132 6587 raft_consensus.cc:2229] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 2 LEADER]: Raft consensus shutting down.
I20260808 13:35:15.639791 6033 raft_consensus.cc:729] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:15.667560 6585 ts_tablet_manager.cc:1905] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:15.662427 6587 raft_consensus.cc:2258] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:15.668144 6033 consensus_queue.cc:260] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.678184 5840 tablet_service.cc:1467] Processing CreateTablet for tablet fcf907a9f19f4d64b27be946011a8ddb (DEFAULT_TABLE table=default.Table_912bf698cf184d5690eeba3080b2b9bf [id=d99ab852cfb94fb4835c8e9c19d293fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.679229 6033 ts_tablet_manager.cc:1428] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.046s user 0.004s sys 0.004s
I20260808 13:35:15.679539 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d6f23c83ff0a4358a7f096203284c706"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:15.680394 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf907a9f19f4d64b27be946011a8ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.684275 5346 tablet_service.cc:1467] Processing CreateTablet for tablet f0ba43d1479a4f9e9d08bdfa29e7157b (DEFAULT_TABLE table=default.Table_8d93597641274e5594c62600af306920 [id=b5733467d1904db5b6df9045484d6a94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.685364 5346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0ba43d1479a4f9e9d08bdfa29e7157b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.686694 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_4ce2d44dea88418eb0e636202d9df995"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:15.688118 4836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b034596e0aa1406ab5a3a849c74a03cc" } modify_external_catalogs: true
I20260808 13:35:15.688611 4836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b034596e0aa1406ab5a3a849c74a03cc" } modify_external_catalogs: true
W20260808 13:35:15.689273 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6f23c83ff0a4358a7f096203284c706 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:15.692252 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ce2d44dea88418eb0e636202d9df995 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:15.681202 5771 consensus_peers.cc:487] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:15.694845 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9bc338093f403881d329a7e913c9f7 (DEFAULT_TABLE table=default.Table_49299116c37c4b9dbb509dbc8efae85e [id=d88dc1106b9b4e1eb18dc680d15fdfcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.696077 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9bc338093f403881d329a7e913c9f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.696383 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_02e12ed060714fadbc5b14d44b2be13c" } new_table_name: "default.Table_43c8f783f487473eb3ea05192f470ef8" modify_external_catalogs: true
I20260808 13:35:15.684010 5841 tablet_service.cc:1467] Processing CreateTablet for tablet f0ba43d1479a4f9e9d08bdfa29e7157b (DEFAULT_TABLE table=default.Table_8d93597641274e5594c62600af306920 [id=b5733467d1904db5b6df9045484d6a94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.698904 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 69642fdba1f145438b04eae9f8154101 (DEFAULT_TABLE table=default.Table_55de3cd70f784a7c90b1028236c8118e [id=e3e121c36ff44ea0a3a98185c8cce381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.686029 6033 tablet_bootstrap.cc:492] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:15.685914 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9bc338093f403881d329a7e913c9f7 (DEFAULT_TABLE table=default.Table_49299116c37c4b9dbb509dbc8efae85e [id=d88dc1106b9b4e1eb18dc680d15fdfcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.700178 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69642fdba1f145438b04eae9f8154101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.704686 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 69642fdba1f145438b04eae9f8154101 (DEFAULT_TABLE table=default.Table_55de3cd70f784a7c90b1028236c8118e [id=e3e121c36ff44ea0a3a98185c8cce381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.704902 5610 tablet_service.cc:1467] Processing CreateTablet for tablet f0ba43d1479a4f9e9d08bdfa29e7157b (DEFAULT_TABLE table=default.Table_8d93597641274e5594c62600af306920 [id=b5733467d1904db5b6df9045484d6a94]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:15.708341 4823 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c -> Peer 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135): Couldn't send request to peer 87e8b888041b4135af55792050defcb5. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260808 13:35:15.711856 6587 ts_tablet_manager.cc:1905] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:15.706223 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9bc338093f403881d329a7e913c9f7 (DEFAULT_TABLE table=default.Table_49299116c37c4b9dbb509dbc8efae85e [id=d88dc1106b9b4e1eb18dc680d15fdfcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.717854 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69642fdba1f145438b04eae9f8154101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.719082 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0ba43d1479a4f9e9d08bdfa29e7157b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.722664 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d13cbde6844079c2d1eb80732bb1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:15.724195 4841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 30) took 1859 ms (client timeout 1000 ms). Trace:
I20260808 13:35:15.726845 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9bc338093f403881d329a7e913c9f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.727916 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 69642fdba1f145438b04eae9f8154101 (DEFAULT_TABLE table=default.Table_55de3cd70f784a7c90b1028236c8118e [id=e3e121c36ff44ea0a3a98185c8cce381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:15.729063 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69642fdba1f145438b04eae9f8154101. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:15.725704 4841 rpcz_store.cc:269] 0808 13:35:13.864960 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:13.977916 (+112956us) service_pool.cc:227] Handling call
0808 13:35:14.104498 (+126582us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:15.724121 (+1619623us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:15.620284 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:15.620632 (+ 348us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:15.620651 (+ 19us) write_op.cc:435] Acquired schema lock
0808 13:35:15.620665 (+ 14us) tablet.cc:660] Decoding operations
0808 13:35:15.620932 (+ 267us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:15.620971 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:15.620993 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:15.621129 (+ 136us) tablet.cc:699] Row locks acquired
0808 13:35:15.621143 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:15.621244 (+ 101us) write_op.cc:270] Start()
0808 13:35:15.621320 (+ 76us) write_op.cc:276] Timestamp: P: 1786196115621164 usec, L: 0
0808 13:35:15.621339 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:15.686570 (+ 65231us) log.cc:842] Serialized 527 byte log entry
0808 13:35:15.687177 (+ 607us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:15.704121 (+ 16944us) op_driver.cc:444] REPLICATION: finished
0808 13:35:15.706960 (+ 2839us) write_op.cc:301] APPLY: starting
0808 13:35:15.707021 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:15.707113 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:15.707129 (+ 16us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:15.707301 (+ 172us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:15.707438 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:15.707487 (+ 49us) write_op.cc:312] APPLY: finished
0808 13:35:15.707655 (+ 168us) log.cc:842] Serialized 23 byte log entry
0808 13:35:15.708263 (+ 608us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:15.708328 (+ 65us) write_op.cc:454] Released schema lock
0808 13:35:15.708362 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":285139328,"child_traces":[["op",{"apply.queue_time_us":2565,"num_ops":1,"prepare.queue_time_us":95,"prepare.run_cpu_time_us":2941,"prepare.run_wall_time_us":67107,"raft.queue_time_us":287,"raft.run_cpu_time_us":594,"raft.run_wall_time_us":594,"replication_time_us":82571,"spinlock_wait_cycles":148892416}]]}
I20260808 13:35:15.774986 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0ba43d1479a4f9e9d08bdfa29e7157b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.819489 6046 tablet_bootstrap.cc:492] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:15.820118 6046 ts_tablet_manager.cc:1397] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.262s user 0.008s sys 0.007s
I20260808 13:35:15.827702 6591 tablet_replica.cc:330] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:15.828729 6467 raft_consensus.cc:482] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:15.829308 6591 raft_consensus.cc:2229] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:15.829191 6467 raft_consensus.cc:504] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.830127 6591 raft_consensus.cc:2258] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:15.841555 6026 tablet_bootstrap.cc:492] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:15.841979 6026 ts_tablet_manager.cc:1397] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.431s user 0.013s sys 0.006s
I20260808 13:35:15.846064 6457 raft_consensus.cc:482] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:15.846778 6457 raft_consensus.cc:504] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:15.848172 6046 raft_consensus.cc:348] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.848981 6046 raft_consensus.cc:374] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:15.849390 6046 raft_consensus.cc:729] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:15.855304 6591 ts_tablet_manager.cc:1905] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:15.860987 6026 raft_consensus.cc:348] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.861354 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9bc338093f403881d329a7e913c9f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:15.861680 6026 raft_consensus.cc:374] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:15.861917 6026 raft_consensus.cc:729] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:15.870330 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:15.873112 5883 raft_consensus.cc:2379] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:15.873402 4840 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
I20260808 13:35:15.878603 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344828b16cfc47d5b419f8b569055fa4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:15.880667 5771 leader_election.cc:341] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:15.862496 6026 consensus_queue.cc:260] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.888841 6026 ts_tablet_manager.cc:1428] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.047s user 0.006s sys 0.002s
I20260808 13:35:15.891031 6593 raft_consensus.cc:482] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:15.891745 6593 raft_consensus.cc:504] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.877602 6467 leader_election.cc:288] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:15.903087 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344828b16cfc47d5b419f8b569055fa4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:15.904080 5644 raft_consensus.cc:2454] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:15.905478 6026 tablet_bootstrap.cc:492] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:15.915027 5772 leader_election.cc:302] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:15.916814 6328 raft_consensus.cc:2781] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:15.917416 6328 raft_consensus.cc:482] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:15.918015 6328 raft_consensus.cc:3037] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:15.850107 6046 consensus_queue.cc:260] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:15.944928 6046 ts_tablet_manager.cc:1428] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.124s user 0.007s sys 0.000s
I20260808 13:35:15.946015 6046 tablet_bootstrap.cc:492] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:15.948208 6026 tablet_bootstrap.cc:654] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:15.974186 6046 tablet_bootstrap.cc:654] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:15.981736 4832 master_service.cc:946] ReplaceTablet: received request to replace tablet 6826e4e4db2043b3a4fe9e7a9359cab6 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:15.988884 6591 ts_tablet_manager.cc:1918] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260808 13:35:15.989645 6591 log.cc:1196] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/8948ce98e7a941f59fa9932fe241638d
I20260808 13:35:15.992515 6593 leader_election.cc:288] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:15.994982 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d388589d386408bb10ca030b87af787" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:15.996106 5644 raft_consensus.cc:2454] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:15.998270 5285 leader_election.cc:302] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:16.001139 6457 leader_election.cc:288] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:16.002067 6457 raft_consensus.cc:2781] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:16.008342 6457 raft_consensus.cc:482] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:16.009089 6457 raft_consensus.cc:3037] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:16.021750 6033 tablet_bootstrap.cc:654] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:16.023464 6046 tablet_bootstrap.cc:492] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:16.023873 6046 ts_tablet_manager.cc:1397] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.078s user 0.012s sys 0.004s
I20260808 13:35:16.037030 6046 raft_consensus.cc:348] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.037851 6046 raft_consensus.cc:374] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:16.038103 6046 raft_consensus.cc:729] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:16.045117 6046 consensus_queue.cc:260] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.055433 6046 ts_tablet_manager.cc:1428] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.031s user 0.008s sys 0.000s
I20260808 13:35:16.056529 6046 tablet_bootstrap.cc:492] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:16.071208 5340 tablet_service.cc:1467] Processing CreateTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e (DEFAULT_TABLE table=default.Table_13f833fb48504c76865676e23461e6e5 [id=0a8dbdef483f48b68019ea98b335a11e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.073448 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.081084 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 9423b380a62c467895e7fdff14756586 (DEFAULT_TABLE table=default.Table_1fe1ec319d834d319cd9c35e801b1c91 [id=0f9219b97bfc4351ad5b584cfdec058f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.082343 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9423b380a62c467895e7fdff14756586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.082227 4835 catalog_manager.cc:5495] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } }
I20260808 13:35:16.087561 4835 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:16.088337 4835 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]). Expected version 1 got 0
I20260808 13:35:16.088910 4835 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (table default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]). Expected version 1 got 0
I20260808 13:35:16.089615 4835 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
I20260808 13:35:16.086292 5346 tablet_service.cc:1467] Processing CreateTablet for tablet b1847d0fecb843cdbc833ad674a503a4 (DEFAULT_TABLE table=default.Table_7bf999991a5c4f998994907960315c0e [id=29db8794594e4221be8974258b5127e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.093808 5346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1847d0fecb843cdbc833ad674a503a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.115429 4832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 6826e4e4db2043b3a4fe9e7a9359cab6
I20260808 13:35:16.116919 4832 catalog_manager.cc:6565] ReplaceTablet: tablet 6826e4e4db2043b3a4fe9e7a9359cab6 deleted and replaced by tablet d8b07b7d31ef41e484389c0f007598f0
I20260808 13:35:16.118098 6593 consensus_queue.cc:1035] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:16.118839 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 with delete_type TABLET_DATA_DELETED (replaced by tablet d8b07b7d31ef41e484389c0f007598f0) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:16.119268 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_f212f0d1d7294b4f80606ba37c7f19f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:16.120764 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f212f0d1d7294b4f80606ba37c7f19f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:16.039640 6601 raft_consensus.cc:482] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:16.121618 6601 raft_consensus.cc:504] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:16.123116 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet bcc0c6ab755c4a6887dabeeef9364201 consensus state has no leader
I20260808 13:35:16.123996 6601 leader_election.cc:288] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:16.080245 5842 tablet_service.cc:1467] Processing CreateTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e (DEFAULT_TABLE table=default.Table_13f833fb48504c76865676e23461e6e5 [id=0a8dbdef483f48b68019ea98b335a11e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.125120 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d388589d386408bb10ca030b87af787" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:16.125898 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.126802 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_347aa48a633d4bf5a0b6767f01742ca9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:16.128190 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_347aa48a633d4bf5a0b6767f01742ca9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:16.085412 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 9423b380a62c467895e7fdff14756586 (DEFAULT_TABLE table=default.Table_1fe1ec319d834d319cd9c35e801b1c91 [id=0f9219b97bfc4351ad5b584cfdec058f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.131331 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9423b380a62c467895e7fdff14756586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.099555 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d388589d386408bb10ca030b87af787" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:16.134846 5882 raft_consensus.cc:2454] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:16.119506 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 with delete_type TABLET_DATA_DELETED (replaced by tablet d8b07b7d31ef41e484389c0f007598f0) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:16.138706 6604 consensus_queue.cc:1035] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:16.139345 5844 tablet_service.cc:1467] Processing CreateTablet for tablet b1847d0fecb843cdbc833ad674a503a4 (DEFAULT_TABLE table=default.Table_7bf999991a5c4f998994907960315c0e [id=29db8794594e4221be8974258b5127e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.140231 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1847d0fecb843cdbc833ad674a503a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.143893 6593 consensus_queue.cc:1035] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:16.158773 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 8514ca242bb2418d8ddbb88548b21832 (DEFAULT_TABLE table=default.Table_280b5d6fc90b4748b675e41e2e7071f5 [id=a2a1f0d168074cabbd4a680e62535962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.160475 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8514ca242bb2418d8ddbb88548b21832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.154922 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 8514ca242bb2418d8ddbb88548b21832 (DEFAULT_TABLE table=default.Table_280b5d6fc90b4748b675e41e2e7071f5 [id=a2a1f0d168074cabbd4a680e62535962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.160063 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 8ba098c946794607b2a15bc64098d918 (DEFAULT_TABLE table=default.Table_2637745cbe454854a74c76f3165627c6 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.163389 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ba098c946794607b2a15bc64098d918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.166906 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8514ca242bb2418d8ddbb88548b21832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.191769 6046 tablet_bootstrap.cc:654] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:16.226403 4832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_4026b16c74ac4327ac4ba5f178bfcaf6" } modify_external_catalogs: true
I20260808 13:35:16.250386 6328 raft_consensus.cc:504] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.260013 6591 ts_tablet_manager.cc:1939] T 8948ce98e7a941f59fa9932fe241638d P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:16.271334 6046 tablet_bootstrap.cc:492] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:16.271785 6046 ts_tablet_manager.cc:1397] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.215s user 0.004s sys 0.015s
I20260808 13:35:16.274252 6046 raft_consensus.cc:348] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.274982 6046 raft_consensus.cc:374] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:16.275257 6046 raft_consensus.cc:729] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:35:16.233599 4837 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 264ee45f8dfa4a05b0e5e812ce1b622c consensus state has no leader
I20260808 13:35:16.275861 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
I20260808 13:35:16.292673 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (table default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]). Expected version 1 got 0
I20260808 13:35:16.293512 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 2 got 0
I20260808 13:35:16.295552 4832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_4026b16c74ac4327ac4ba5f178bfcaf6" } modify_external_catalogs: true
I20260808 13:35:16.267688 6601 raft_consensus.cc:482] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:16.278503 6033 tablet_bootstrap.cc:492] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
W20260808 13:35:16.284730 5835 consensus_meta.cc:287] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.668s user 0.004s sys 0.006s
I20260808 13:35:16.294963 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]). Expected version 1 got 0
I20260808 13:35:16.327257 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d388589d386408bb10ca030b87af787" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:16.328269 5640 raft_consensus.cc:2454] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:16.325897 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:16.346251 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
I20260808 13:35:16.348482 5340 tablet_service.cc:1467] Processing CreateTablet for tablet d0cc688386f3413ea939dda3a75d6541 (DEFAULT_TABLE table=default.Table_e37ea7df0fbb4dea827afe2023a0fac5 [id=452cc67848cf43efb3d80e76fdca3b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.348129 4841 catalog_manager.cc:5495] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:16.349855 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 2 got 0
I20260808 13:35:16.350967 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (table default.Table_60e5f930145b4e7db14190e5896be3f4 [id=19a3a9ca647c4e069a20960c7f4cc57a]). Expected version 1 got 0
I20260808 13:35:16.351882 4841 catalog_manager.cc:5495] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:16.352656 5611 tablet_service.cc:1467] Processing CreateTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e (DEFAULT_TABLE table=default.Table_13f833fb48504c76865676e23461e6e5 [id=0a8dbdef483f48b68019ea98b335a11e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.353302 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:16.354544 6587 ts_tablet_manager.cc:1918] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260808 13:35:16.355185 6587 log.cc:1196] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/8948ce98e7a941f59fa9932fe241638d
I20260808 13:35:16.355628 4833 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 162) took 1857 ms. Trace:
I20260808 13:35:16.355528 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 9423b380a62c467895e7fdff14756586 (DEFAULT_TABLE table=default.Table_1fe1ec319d834d319cd9c35e801b1c91 [id=0f9219b97bfc4351ad5b584cfdec058f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.359366 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 with delete_type TABLET_DATA_DELETED (replaced by tablet d8b07b7d31ef41e484389c0f007598f0) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:16.359926 5614 tablet_service.cc:1467] Processing CreateTablet for tablet b1847d0fecb843cdbc833ad674a503a4 (DEFAULT_TABLE table=default.Table_7bf999991a5c4f998994907960315c0e [id=29db8794594e4221be8974258b5127e0]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:16.342320 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 82275.8us
I20260808 13:35:16.363291 6591 tablet_replica.cc:330] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:16.355983 4833 rpcz_store.cc:276] 0808 13:35:14.498747 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:14.498913 (+ 166us) service_pool.cc:227] Handling call
0808 13:35:14.511001 (+ 12088us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff7d6 00007f227bd45634 00007f227bd455c2 00007f227bd454f1 00007f227bd4546c 00007f227bd44722 00007f2279fde94c 00007f2279fddd86
0808 13:35:14.515840 (+ 4839us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:14.519061 (+ 3221us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:14.519760 (+ 699us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:14.519945 (+ 185us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:15.129303 (+609358us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:16.355548 (+1226245us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:14.574254 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:14.610865 (+ 36611us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:14.610888 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:14.610900 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:14.611188 (+ 288us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:14.616626 (+ 5438us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:14.616661 (+ 35us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:14.616820 (+ 159us) tablet.cc:699] Row locks acquired
0808 13:35:14.616835 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:14.616949 (+ 114us) write_op.cc:270] Start()
0808 13:35:14.617030 (+ 81us) write_op.cc:276] Timestamp: P: 1786196114616870 usec, L: 0
0808 13:35:14.617047 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:14.849501 (+232454us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b5400022018. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227cf570c4 00007f227c2034f9 00007f227c1bef91 00007f227c1bec6a 00007f227c22acd5 00007f227c22b179 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9
0808 13:35:14.849907 (+ 406us) log.cc:842] Serialized 405 byte log entry
0808 13:35:14.850400 (+ 493us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:15.091417 (+241017us) op_driver.cc:444] REPLICATION: finished
0808 13:35:15.091812 (+ 395us) write_op.cc:301] APPLY: starting
0808 13:35:15.091861 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:15.091953 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:15.091972 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:15.092276 (+ 304us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b58000c05b0. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c1a9834 00007f227c1ae771 00007f227c1ae6f9 00007f227c1ae689 00007f227c1ae651 00007f227c1ad81d 0000000000580784 00000000005805b9 00007f22752da6b6 00007f22752ddb11
0808 13:35:15.125286 (+ 33010us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:15.128205 (+ 2919us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:15.128285 (+ 80us) write_op.cc:312] APPLY: finished
0808 13:35:15.128506 (+ 221us) log.cc:842] Serialized 23 byte log entry
0808 13:35:15.128671 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:15.128738 (+ 67us) write_op.cc:454] Released schema lock
0808 13:35:15.128771 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:15.128966 (+ 195us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:15.128987 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":24648960,"child_traces":[["op",{"apply.queue_time_us":189,"apply.run_cpu_time_us":1712,"apply.run_wall_time_us":37353,"num_ops":1,"prepare.queue_time_us":52913,"prepare.run_cpu_time_us":5879,"prepare.run_wall_time_us":280013,"raft.queue_time_us":3174,"raft.run_cpu_time_us":2172,"raft.run_wall_time_us":238590,"replication_time_us":474293,"spinlock_wait_cycles":572253568}]]}
I20260808 13:35:16.365326 6033 ts_tablet_manager.cc:1397] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.679s user 0.012s sys 0.004s
I20260808 13:35:16.366683 6026 tablet_bootstrap.cc:492] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:16.367108 6026 ts_tablet_manager.cc:1397] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.462s user 0.007s sys 0.014s
I20260808 13:35:16.368429 6587 ts_tablet_manager.cc:1939] T 8948ce98e7a941f59fa9932fe241638d P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:16.369606 6046 consensus_queue.cc:260] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.378590 6046 ts_tablet_manager.cc:1428] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.107s user 0.008s sys 0.000s
I20260808 13:35:16.380591 6046 tablet_bootstrap.cc:492] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:16.381592 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0cc688386f3413ea939dda3a75d6541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.382231 6026 raft_consensus.cc:348] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.383047 6026 raft_consensus.cc:374] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:16.385336 6026 raft_consensus.cc:729] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:16.386008 6026 consensus_queue.cc:260] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.386920 4840 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4] alter complete (version 1)
I20260808 13:35:16.391168 6457 raft_consensus.cc:504] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.393302 4836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet edfa66acceea4b54911d78ee16609ef7
I20260808 13:35:16.394030 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.396618 5383 raft_consensus.cc:2379] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:16.399149 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9423b380a62c467895e7fdff14756586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.401504 6457 leader_election.cc:288] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:16.403328 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1847d0fecb843cdbc833ad674a503a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.415082 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]) successfully deleted
W20260808 13:35:16.416702 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 45a26598a69146e1950e9227523fff13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45a26598a69146e1950e9227523fff13 already in progress: creating tablet
W20260808 13:35:16.417806 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6826e4e4db2043b3a4fe9e7a9359cab6 already in progress: creating tablet
I20260808 13:35:16.418057 6026 ts_tablet_manager.cc:1428] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.051s user 0.005s sys 0.003s
I20260808 13:35:16.422920 6026 tablet_bootstrap.cc:492] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:16.421401 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c (DEFAULT_TABLE table=default.Table_43c8f783f487473eb3ea05192f470ef8 [id=b421a6c36f8e42ca96490c98ed957ccd]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:16.410527 4839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 21) took 2909 ms (client timeout 1000 ms). Trace:
W20260808 13:35:16.430236 4839 rpcz_store.cc:269] 0808 13:35:13.501492 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:13.502263 (+ 771us) service_pool.cc:227] Handling call
0808 13:35:13.511539 (+ 9276us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:16.410433 (+2898894us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:15.567123 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:15.567484 (+ 361us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:15.567509 (+ 25us) write_op.cc:435] Acquired schema lock
0808 13:35:15.567521 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:15.567963 (+ 442us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:15.567997 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:15.568018 (+ 21us) tablet.cc:683] Acquiring locks for 3 operations
0808 13:35:15.568258 (+ 240us) tablet.cc:699] Row locks acquired
0808 13:35:15.568271 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:15.568383 (+ 112us) write_op.cc:270] Start()
0808 13:35:15.568466 (+ 83us) write_op.cc:276] Timestamp: P: 1786196115568303 usec, L: 0
0808 13:35:15.568482 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:15.569007 (+ 525us) log.cc:842] Serialized 1275 byte log entry
0808 13:35:15.600053 (+ 31046us) op_driver.cc:444] REPLICATION: finished
0808 13:35:15.609665 (+ 9612us) write_op.cc:301] APPLY: starting
0808 13:35:15.609716 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:15.609874 (+ 158us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:15.609984 (+ 110us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:15.610490 (+ 506us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:15.610686 (+ 196us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0808 13:35:15.610749 (+ 63us) write_op.cc:312] APPLY: finished
0808 13:35:15.610977 (+ 228us) log.cc:842] Serialized 35 byte log entry
0808 13:35:15.611629 (+ 652us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:15.611757 (+ 128us) write_op.cc:454] Released schema lock
0808 13:35:15.611823 (+ 66us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:15.693343 (+ 81520us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:15.693375 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":10,"spinlock_wait_cycles":14670592,"child_traces":[["op",{"apply.queue_time_us":9485,"apply.run_cpu_time_us":2099,"apply.run_wall_time_us":83916,"num_ops":3,"prepare.queue_time_us":214726,"prepare.run_cpu_time_us":2217,"prepare.run_wall_time_us":30845,"raft.queue_time_us":109,"raft.run_cpu_time_us":1094,"raft.run_wall_time_us":1140,"replication_time_us":31497,"thread_start_us":77613,"threads_started":1}]]}
I20260808 13:35:16.433637 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264ee45f8dfa4a05b0e5e812ce1b622c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.434446 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet edfa66acceea4b54911d78ee16609ef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:15 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:16.436427 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 16a6552c871b4b81b6520cb084d359a2 (DEFAULT_TABLE table=default.Table_eeada3d5a8ba4dc4b96d74955c23f2bc [id=2fa9c2e807544cc8916c41200a0691cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.438681 6587 tablet_replica.cc:330] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:16.438798 6591 raft_consensus.cc:2229] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:16.440054 6587 raft_consensus.cc:2229] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:16.440847 6591 pending_rounds.cc:62] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: Trying to abort 1 pending ops.
I20260808 13:35:16.440796 6587 raft_consensus.cc:2258] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:16.441113 6591 pending_rounds.cc:68] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259286343913472 op_type: NO_OP noop_request { }
I20260808 13:35:16.441598 6591 raft_consensus.cc:2866] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:35:16.442034 6591 raft_consensus.cc:2258] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:16.445415 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16a6552c871b4b81b6520cb084d359a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.452243 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 8514ca242bb2418d8ddbb88548b21832 (DEFAULT_TABLE table=default.Table_280b5d6fc90b4748b675e41e2e7071f5 [id=a2a1f0d168074cabbd4a680e62535962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.453915 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8514ca242bb2418d8ddbb88548b21832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.438488 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_920e3ee624f44ee4b19c15e0d7fb4358"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:16.455814 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_920e3ee624f44ee4b19c15e0d7fb4358 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:16.456007 6587 ts_tablet_manager.cc:1905] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:16.465497 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 8ba098c946794607b2a15bc64098d918 (DEFAULT_TABLE table=default.Table_2637745cbe454854a74c76f3165627c6 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.466576 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ba098c946794607b2a15bc64098d918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.465799 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d70de767ab2548b4a9e45418eefc9a2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:16.468905 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d70de767ab2548b4a9e45418eefc9a2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:16.471323 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_7d31aaee9dd04cabbbbca675bde3a2f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:16.472788 6046 tablet_bootstrap.cc:654] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:16.472810 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d31aaee9dd04cabbbbca675bde3a2f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:16.477809 5603 tablet_service.cc:1467] Processing CreateTablet for tablet d0cc688386f3413ea939dda3a75d6541 (DEFAULT_TABLE table=default.Table_e37ea7df0fbb4dea827afe2023a0fac5 [id=452cc67848cf43efb3d80e76fdca3b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.479885 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0cc688386f3413ea939dda3a75d6541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.492272 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 8ba098c946794607b2a15bc64098d918 (DEFAULT_TABLE table=default.Table_2637745cbe454854a74c76f3165627c6 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.494781 5772 leader_election.cc:302] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:16.502840 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d388589d386408bb10ca030b87af787" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:16.503454 5835 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 94) took 1161 ms. Trace:
I20260808 13:35:16.503902 5881 raft_consensus.cc:3037] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:16.504016 5835 rpcz_store.cc:276] 0808 13:35:15.342068 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:15.342223 (+ 155us) service_pool.cc:227] Handling call
0808 13:35:15.343637 (+ 1414us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:15.343731 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:15.616360 (+272629us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:16.485592 (+869232us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:16.503349 (+ 17757us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:16.512459 6328 leader_election.cc:288] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:16.516989 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c (DEFAULT_TABLE table=default.Table_43c8f783f487473eb3ea05192f470ef8 [id=b421a6c36f8e42ca96490c98ed957ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.518987 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264ee45f8dfa4a05b0e5e812ce1b622c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.520943 6033 raft_consensus.cc:348] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.521664 6033 raft_consensus.cc:374] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:16.521927 6033 raft_consensus.cc:729] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:16.523572 5609 tablet_service.cc:1514] Processing DeleteTablet for tablet edfa66acceea4b54911d78ee16609ef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:15 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:16.524802 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet edfa66acceea4b54911d78ee16609ef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:15 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:16.525170 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c (DEFAULT_TABLE table=default.Table_43c8f783f487473eb3ea05192f470ef8 [id=b421a6c36f8e42ca96490c98ed957ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.525665 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d492406c8a4c4186131e02a1559cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:16.524080 6617 consensus_queue.cc:1035] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:16.366094 6601 raft_consensus.cc:504] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:16.531793 6601 leader_election.cc:288] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:16.534814 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ba098c946794607b2a15bc64098d918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.552273 6591 ts_tablet_manager.cc:1905] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:16.564239 6026 tablet_bootstrap.cc:654] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:16.570956 4838 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_347aa48a633d4bf5a0b6767f01742ca9" } new_table_name: "default.Table_7da887b39f244cf0b44cbe9fa3b0eae2" modify_external_catalogs: true
I20260808 13:35:16.573292 5609 tablet_service.cc:1467] Processing CreateTablet for tablet 16a6552c871b4b81b6520cb084d359a2 (DEFAULT_TABLE table=default.Table_eeada3d5a8ba4dc4b96d74955c23f2bc [id=2fa9c2e807544cc8916c41200a0691cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.574445 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16a6552c871b4b81b6520cb084d359a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.522555 6033 consensus_queue.cc:260] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.580914 4840 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_fd7c3f5b3751428695967c6657d567d6 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7] alter complete (version 1)
W20260808 13:35:16.584878 4840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 38) took 1992 ms (client timeout 1000 ms). Trace:
I20260808 13:35:16.587476 4832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 9260370cf8cb46a7b699581e4e8d46db
I20260808 13:35:16.592203 5356 tablet_service.cc:1467] Processing CreateTablet for tablet f66b38e83dab4bf1a3d2d1e088a96988 (DEFAULT_TABLE table=default.Table_d30a1f7564df4e6085a1858c6f3924fc [id=4296a36ca9b74888aeccfe22e3970de6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.592492 6033 ts_tablet_manager.cc:1428] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.227s user 0.010s sys 0.002s
I20260808 13:35:16.556087 5835 tablet_service.cc:1467] Processing CreateTablet for tablet d0cc688386f3413ea939dda3a75d6541 (DEFAULT_TABLE table=default.Table_e37ea7df0fbb4dea827afe2023a0fac5 [id=452cc67848cf43efb3d80e76fdca3b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.593441 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66b38e83dab4bf1a3d2d1e088a96988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.595393 5834 tablet_service.cc:1467] Processing CreateTablet for tablet 16a6552c871b4b81b6520cb084d359a2 (DEFAULT_TABLE table=default.Table_eeada3d5a8ba4dc4b96d74955c23f2bc [id=2fa9c2e807544cc8916c41200a0691cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.596966 5607 tablet_service.cc:1467] Processing CreateTablet for tablet f66b38e83dab4bf1a3d2d1e088a96988 (DEFAULT_TABLE table=default.Table_d30a1f7564df4e6085a1858c6f3924fc [id=4296a36ca9b74888aeccfe22e3970de6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.598073 5833 tablet_service.cc:1514] Processing DeleteTablet for tablet 9260370cf8cb46a7b699581e4e8d46db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:16.598490 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66b38e83dab4bf1a3d2d1e088a96988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.601212 5346 tablet_service.cc:1514] Processing DeleteTablet for tablet 9260370cf8cb46a7b699581e4e8d46db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:16.602936 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 31) took 1506 ms (client timeout 1000 ms). Trace:
I20260808 13:35:16.603417 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 9260370cf8cb46a7b699581e4e8d46db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:16.603863 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:16.604259 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:16.605053 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:16.603277 4835 rpcz_store.cc:269] 0808 13:35:15.096818 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:15.227522 (+130704us) service_pool.cc:227] Handling call
0808 13:35:15.299522 (+ 72000us) spinlock_profiling.cc:243] Waited 68.5 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:16.602845 (+1303323us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:16.214822 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:16.215254 (+ 432us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:16.215276 (+ 22us) write_op.cc:435] Acquired schema lock
0808 13:35:16.215288 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:16.215559 (+ 271us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:16.215608 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:16.215627 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:16.215767 (+ 140us) tablet.cc:699] Row locks acquired
0808 13:35:16.215781 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:16.215885 (+ 104us) write_op.cc:270] Start()
0808 13:35:16.215960 (+ 75us) write_op.cc:276] Timestamp: P: 1786196116215671 usec, L: 0
0808 13:35:16.215977 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:16.216504 (+ 527us) log.cc:842] Serialized 527 byte log entry
0808 13:35:16.247494 (+ 30990us) op_driver.cc:444] REPLICATION: finished
0808 13:35:16.247966 (+ 472us) write_op.cc:301] APPLY: starting
0808 13:35:16.248028 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:16.248128 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:16.248221 (+ 93us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:16.248394 (+ 173us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:16.248632 (+ 238us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:16.248743 (+ 111us) write_op.cc:312] APPLY: finished
0808 13:35:16.248978 (+ 235us) log.cc:842] Serialized 23 byte log entry
0808 13:35:16.249691 (+ 713us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:16.249820 (+ 129us) write_op.cc:454] Released schema lock
0808 13:35:16.249894 (+ 74us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:16.250217 (+ 323us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:16.250278 (+ 61us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":15,"spinlock_wait_cycles":157607808,"child_traces":[["op",{"apply.queue_time_us":326,"apply.run_cpu_time_us":2042,"apply.run_wall_time_us":2510,"num_ops":1,"prepare.queue_time_us":177,"prepare.run_cpu_time_us":1958,"prepare.run_wall_time_us":10502,"raft.queue_time_us":229,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":11,"replication_time_us":31416}]]}
I20260808 13:35:16.609041 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344828b16cfc47d5b419f8b569055fa4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:16.611665 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:16.612434 5378 raft_consensus.cc:2454] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
W20260808 13:35:16.585197 4840 rpcz_store.cc:269] 0808 13:35:14.592791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:14.592951 (+ 160us) service_pool.cc:227] Handling call
0808 13:35:16.584812 (+1991861us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:15.877987 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:15.878480 (+ 493us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:15.878514 (+ 34us) write_op.cc:435] Acquired schema lock
0808 13:35:15.878526 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:15.878931 (+ 405us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:15.879023 (+ 92us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:15.879056 (+ 33us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:15.879222 (+ 166us) tablet.cc:699] Row locks acquired
0808 13:35:15.879235 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:15.879363 (+ 128us) write_op.cc:270] Start()
0808 13:35:15.879458 (+ 95us) write_op.cc:276] Timestamp: P: 1786196115879173 usec, L: 0
0808 13:35:15.879474 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:15.880088 (+ 614us) log.cc:842] Serialized 356 byte log entry
0808 13:35:15.897952 (+ 17864us) op_driver.cc:444] REPLICATION: finished
0808 13:35:15.898468 (+ 516us) write_op.cc:301] APPLY: starting
0808 13:35:15.898588 (+ 120us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:15.898767 (+ 179us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:15.898936 (+ 169us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:15.899193 (+ 257us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:15.899462 (+ 269us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:15.899552 (+ 90us) write_op.cc:312] APPLY: finished
0808 13:35:15.899838 (+ 286us) log.cc:842] Serialized 23 byte log entry
0808 13:35:15.900098 (+ 260us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:15.900224 (+ 126us) write_op.cc:454] Released schema lock
0808 13:35:15.900281 (+ 57us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:15.900593 (+ 312us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:15.900669 (+ 76us) write_op.cc:454] Released schema lock
Related trace 'op':
0808 13:35:16.512207 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:16.512602 (+ 395us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:16.512634 (+ 32us) write_op.cc:435] Acquired schema lock
0808 13:35:16.512649 (+ 15us) tablet.cc:660] Decoding operations
0808 13:35:16.513036 (+ 387us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:16.513082 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:16.513103 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:16.513259 (+ 156us) tablet.cc:699] Row locks acquired
0808 13:35:16.513272 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:16.513412 (+ 140us) write_op.cc:270] Start()
0808 13:35:16.513517 (+ 105us) write_op.cc:276] Timestamp: P: 1786196116513325 usec, L: 0
0808 13:35:16.513532 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:16.514025 (+ 493us) log.cc:842] Serialized 356 byte log entry
0808 13:35:16.577461 (+ 63436us) op_driver.cc:444] REPLICATION: finished
0808 13:35:16.579438 (+ 1977us) write_op.cc:301] APPLY: starting
0808 13:35:16.579486 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:16.579572 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:16.579655 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:16.579791 (+ 136us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:16.579911 (+ 120us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:16.579952 (+ 41us) write_op.cc:312] APPLY: finished
0808 13:35:16.580095 (+ 143us) log.cc:842] Serialized 23 byte log entry
0808 13:35:16.580208 (+ 113us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:16.580259 (+ 51us) write_op.cc:454] Released schema lock
0808 13:35:16.580286 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:16.580457 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:16.580478 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":239,"apply.run_cpu_time_us":2480,"apply.run_wall_time_us":2479,"num_ops":1,"prepare.queue_time_us":107,"prepare.run_cpu_time_us":2384,"prepare.run_wall_time_us":2386,"raft.queue_time_us":183,"raft.run_cpu_time_us":1447,"raft.run_wall_time_us":1446,"replication_time_us":18364}],["op",{"apply.queue_time_us":1832,"apply.run_cpu_time_us":1167,"apply.run_wall_time_us":1167,"num_ops":1,"prepare.queue_time_us":122741,"prepare.run_cpu_time_us":2147,"prepare.run_wall_time_us":54691,"raft.queue_time_us":3502,"replication_time_us":63854}]]}
I20260808 13:35:16.613278 6585 ts_tablet_manager.cc:1918] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260808 13:35:16.613699 6585 log.cc:1196] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/8948ce98e7a941f59fa9932fe241638d
I20260808 13:35:16.616055 6601 raft_consensus.cc:482] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:16.616510 6601 raft_consensus.cc:504] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.618510 6601 leader_election.cc:288] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:16.619374 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d13cbde6844079c2d1eb80732bb1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:16.620785 4839 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fd7c3f5b3751428695967c6657d567d6" } new_table_name: "default.Table_fd5435dd94cd493ab4780d4dea9de9a8" modify_external_catalogs: true
I20260808 13:35:16.624114 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c3535f396f74749a6a28907c6d93abd" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:16.631788 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:16.635519 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d388589d386408bb10ca030b87af787" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:16.636307 5637 raft_consensus.cc:3037] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:16.637673 5881 raft_consensus.cc:2454] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:16.638254 6485 raft_consensus.cc:2741] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260808 13:35:16.647001 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 152325us
W20260808 13:35:16.647532 5771 leader_election.cc:341] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:16.648511 5771 leader_election.cc:302] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:16.652410 6328 raft_consensus.cc:2781] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:35:16.652935 6328 raft_consensus.cc:482] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:16.653409 6328 raft_consensus.cc:3037] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:16.653945 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344828b16cfc47d5b419f8b569055fa4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:16.654763 5644 raft_consensus.cc:3037] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:16.659711 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb8c7eeec164485a4e5bb80f9d5fd72 (DEFAULT_TABLE table=default.Table_fc33bfcdedd744039ea0ff6c698d01cc [id=8e1d239d7f7446fa89184bf11ddb88d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.664090 6328 raft_consensus.cc:504] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:16.665059 5644 raft_consensus.cc:2454] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:16.665866 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c3535f396f74749a6a28907c6d93abd" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:16.675362 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_1eb4e0870a824768a3ecace6bf383c25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:16.677381 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1eb4e0870a824768a3ecace6bf383c25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:16.677541 5346 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb8c7eeec164485a4e5bb80f9d5fd72 (DEFAULT_TABLE table=default.Table_fc33bfcdedd744039ea0ff6c698d01cc [id=8e1d239d7f7446fa89184bf11ddb88d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.687602 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_23396b27c08444c7981e55a66139f6b6" } modify_external_catalogs: true
I20260808 13:35:16.688519 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_23396b27c08444c7981e55a66139f6b6" } modify_external_catalogs: true
I20260808 13:35:16.694864 5346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb8c7eeec164485a4e5bb80f9d5fd72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.666083 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264ee45f8dfa4a05b0e5e812ce1b622c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:16.709698 4842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 22) took 1630 ms (client timeout 1000 ms). Trace:
W20260808 13:35:16.710145 4842 rpcz_store.cc:269] 0808 13:35:15.079015 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:15.221902 (+142887us) service_pool.cc:227] Handling call
0808 13:35:15.281114 (+ 59212us) spinlock_profiling.cc:243] Waited 55.2 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:16.709629 (+1428515us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":16,"spinlock_wait_cycles":126991104}
I20260808 13:35:16.715749 5637 raft_consensus.cc:2454] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:16.725142 6026 tablet_bootstrap.cc:492] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:16.725617 6026 ts_tablet_manager.cc:1397] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.303s user 0.014s sys 0.000s
I20260808 13:35:16.728235 6026 raft_consensus.cc:348] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.733301 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7284afe0294644a368cf51451c7ab6 (DEFAULT_TABLE table=default.Table_9b3bbebdc1274b238c30a45d36cf86f7 [id=26fb789fff434bb0825daf68d5378013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.735199 6026 raft_consensus.cc:374] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:16.735482 6026 raft_consensus.cc:729] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:16.735527 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0cc688386f3413ea939dda3a75d6541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.736093 6457 raft_consensus.cc:482] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:16.736039 6026 consensus_queue.cc:260] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.736567 6457 raft_consensus.cc:504] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.739449 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16a6552c871b4b81b6520cb084d359a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.728160 5838 tablet_service.cc:1467] Processing CreateTablet for tablet f66b38e83dab4bf1a3d2d1e088a96988 (DEFAULT_TABLE table=default.Table_d30a1f7564df4e6085a1858c6f3924fc [id=4296a36ca9b74888aeccfe22e3970de6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.750082 6026 ts_tablet_manager.cc:1428] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.024s user 0.003s sys 0.002s
I20260808 13:35:16.750885 6026 tablet_bootstrap.cc:492] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:16.754788 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7284afe0294644a368cf51451c7ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.750505 6485 consensus_queue.cc:1035] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260808 13:35:16.765110 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66b38e83dab4bf1a3d2d1e088a96988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.771250 5840 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 99) took 1186 ms. Trace:
I20260808 13:35:16.774331 5840 rpcz_store.cc:276] 0808 13:35:15.585028 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:15.677431 (+ 92403us) service_pool.cc:227] Handling call
0808 13:35:15.679931 (+ 2500us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:15.680030 (+ 99us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:16.226072 (+546042us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:16.611727 (+385655us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:16.764389 (+152662us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b5800001898. stack: 00007f19df247033 00007f19df246f30 00007f19dedc8534 00000000004db2b4 00000000004db249 00007f19df0bdf11 00007f19df241851 00007f19df241fee 00007f19e81018f9 00007f19e71356a1 00007f19e71f920b 00007f19e71f4e2f 00007f19e71f55ed 00007f19e71a6869 00007f19e5898c44 00007f19e5898bd2
0808 13:35:16.771187 (+ 6798us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2360064}
I20260808 13:35:16.773108 6033 tablet_bootstrap.cc:492] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:16.773979 5841 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 103) took 1091 ms. Trace:
I20260808 13:35:16.775816 5841 rpcz_store.cc:276] 0808 13:35:15.683011 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:15.683328 (+ 317us) service_pool.cc:227] Handling call
0808 13:35:15.699002 (+ 15674us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:15.699081 (+ 79us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:16.272066 (+572985us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:16.664043 (+391977us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:16.773919 (+109876us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:16.772205 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb8c7eeec164485a4e5bb80f9d5fd72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.777068 6457 leader_election.cc:288] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:16.756150 6585 ts_tablet_manager.cc:1939] T 8948ce98e7a941f59fa9932fe241638d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:16.787551 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 56d8d5a3ca724141b58e4e2cea23be68 (DEFAULT_TABLE table=default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.779655 5355 tablet_service.cc:1467] Processing CreateTablet for tablet d9268f5f6dee4112932e106010a3c2be (DEFAULT_TABLE table=default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.789094 4842 master_service.cc:946] ReplaceTablet: received request to replace tablet a0b29786cb404bc2b01473c3b8390a50 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:16.789315 6033 tablet_bootstrap.cc:654] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:16.789233 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d8d5a3ca724141b58e4e2cea23be68. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:16.744235 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 95895.2us
I20260808 13:35:16.792989 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_13f833fb48504c76865676e23461e6e5" } modify_external_catalogs: true
I20260808 13:35:16.793543 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_13f833fb48504c76865676e23461e6e5" } modify_external_catalogs: true
I20260808 13:35:16.790723 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:16.793774 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 100) took 1194 ms. Trace:
W20260808 13:35:16.754825 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 85695us
I20260808 13:35:16.804188 6626 tablet.cc:1722] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:16.812480 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]) successfully deleted
W20260808 13:35:16.813436 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 45a26598a69146e1950e9227523fff13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45a26598a69146e1950e9227523fff13 already in progress: creating tablet
W20260808 13:35:16.814921 5285 leader_election.cc:341] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 78d492406c8a4c4186131e02a1559cb4
I20260808 13:35:16.815389 5285 leader_election.cc:302] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:16.816452 6457 raft_consensus.cc:3037] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:16.821352 6593 consensus_queue.cc:1035] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:16.822587 6593 consensus_queue.cc:1035] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:16.827540 5771 leader_election.cc:341] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:16.830940 5383 raft_consensus.cc:3037] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:16.802507 6585 rpcz_store.cc:276] 0808 13:35:15.599343 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:15.599496 (+ 153us) service_pool.cc:227] Handling call
0808 13:35:15.637928 (+ 38432us) ts_tablet_manager.cc:1145] Deleting tablet 8948ce98e7a941f59fa9932fe241638d
0808 13:35:15.637977 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:16.268332 (+630355us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:16.613005 (+344673us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:16.793710 (+180705us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":37809,"thread_start_us":37286,"threads_started":1}
I20260808 13:35:16.833276 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 101) took 1222 ms. Trace:
I20260808 13:35:16.833767 6585 rpcz_store.cc:276] 0808 13:35:15.611390 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:15.611552 (+ 162us) service_pool.cc:227] Handling call
0808 13:35:16.832861 (+1221309us) ts_tablet_manager.cc:1145] Deleting tablet 45a26598a69146e1950e9227523fff13
0808 13:35:16.832904 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:16.833218 (+ 314us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1220628}
I20260808 13:35:16.835489 6585 tablet_replica.cc:330] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:16.836294 6585 raft_consensus.cc:2229] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:16.839226 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9268f5f6dee4112932e106010a3c2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.844506 6046 tablet_bootstrap.cc:492] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:16.848426 6046 ts_tablet_manager.cc:1397] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.468s user 0.007s sys 0.013s
I20260808 13:35:16.861302 6457 raft_consensus.cc:2726] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:16.824391 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:16.865360 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 8948ce98e7a941f59fa9932fe241638d (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]) successfully deleted
W20260808 13:35:16.867187 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 45a26598a69146e1950e9227523fff13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45a26598a69146e1950e9227523fff13 already in progress: creating tablet
I20260808 13:35:16.867605 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
W20260808 13:35:16.867758 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6826e4e4db2043b3a4fe9e7a9359cab6 already in progress: creating tablet
I20260808 13:35:16.868491 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet e2a95712de254c13b3b6dc4fe5cb15f7 (table default.Table_0a3697cdad3740a1baa47b3075b5e1f9 [id=19a3a9ca647c4e069a20960c7f4cc57a]). Expected version 2 got 0
I20260808 13:35:16.869136 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:16.871516 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6e898d8b2bda40a8b235d59bc101c5e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:16.873232 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6e898d8b2bda40a8b235d59bc101c5e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:16.874477 4838 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f85b4fd30d6c4903946891a399acf7a0 has no consensus state
I20260808 13:35:16.862028 6046 raft_consensus.cc:348] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.872197 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d492406c8a4c4186131e02a1559cb4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:16.878891 5382 raft_consensus.cc:3037] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:16.879009 4838 master_service.cc:946] ReplaceTablet: received request to replace tablet e2a95712de254c13b3b6dc4fe5cb15f7 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:16.888346 5382 raft_consensus.cc:2454] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 2.
I20260808 13:35:16.892422 6046 raft_consensus.cc:374] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:16.893646 6046 raft_consensus.cc:729] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:16.894773 6046 consensus_queue.cc:260] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:16.916087 5609 tablet_service.cc:1467] Processing CreateTablet for tablet 56d8d5a3ca724141b58e4e2cea23be68 (DEFAULT_TABLE table=default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.917811 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d8d5a3ca724141b58e4e2cea23be68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.920143 6046 ts_tablet_manager.cc:1428] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.071s user 0.004s sys 0.001s
I20260808 13:35:16.843103 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7284afe0294644a368cf51451c7ab6 (DEFAULT_TABLE table=default.Table_9b3bbebdc1274b238c30a45d36cf86f7 [id=26fb789fff434bb0825daf68d5378013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.844581 6328 leader_election.cc:288] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:16.845083 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:16.854156 5845 tablet_service.cc:1467] Processing CreateTablet for tablet d9268f5f6dee4112932e106010a3c2be (DEFAULT_TABLE table=default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:16.861759 5772 leader_election.cc:341] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 78d492406c8a4c4186131e02a1559cb4
I20260808 13:35:16.863134 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 56d8d5a3ca724141b58e4e2cea23be68 (DEFAULT_TABLE table=default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:16.923954 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 62235.9us
I20260808 13:35:16.924853 5772 leader_election.cc:302] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:16.926045 5772 leader_election.cc:341] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:16.927515 5772 leader_election.cc:302] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:16.926558 6328 raft_consensus.cc:2781] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:16.928117 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7284afe0294644a368cf51451c7ab6 (DEFAULT_TABLE table=default.Table_9b3bbebdc1274b238c30a45d36cf86f7 [id=26fb789fff434bb0825daf68d5378013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.928368 6328 raft_consensus.cc:686] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:16.929340 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7284afe0294644a368cf51451c7ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.929123 6328 consensus_queue.cc:237] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:16.873157 5771 consensus_peers.cc:487] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:16.932986 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 59943.1us
I20260808 13:35:16.935117 5771 leader_election.cc:302] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:16.936978 5613 tablet_service.cc:1467] Processing CreateTablet for tablet d9268f5f6dee4112932e106010a3c2be (DEFAULT_TABLE table=default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.938151 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9268f5f6dee4112932e106010a3c2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.939683 6622 raft_consensus.cc:2781] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:35:16.942730 6622 raft_consensus.cc:686] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:16.942893 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb8c7eeec164485a4e5bb80f9d5fd72 (DEFAULT_TABLE table=default.Table_fc33bfcdedd744039ea0ff6c698d01cc [id=8e1d239d7f7446fa89184bf11ddb88d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:16.944013 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb8c7eeec164485a4e5bb80f9d5fd72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.943545 6622 consensus_queue.cc:237] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:16.946408 6046 tablet_bootstrap.cc:492] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:16.949836 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7284afe0294644a368cf51451c7ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:16.959909 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d13cbde6844079c2d1eb80732bb1f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:16.962713 6585 raft_consensus.cc:2258] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:16.964555 6585 ts_tablet_manager.cc:1905] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:16.968662 5283 leader_election.cc:302] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:16.977706 6046 tablet_bootstrap.cc:654] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:16.978325 6601 raft_consensus.cc:2726] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:16.980089 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 45314.3us
I20260808 13:35:16.990021 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_661e11184b9346b1bbae005034dddd62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:16.991976 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_661e11184b9346b1bbae005034dddd62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:16.932106 6640 raft_consensus.cc:482] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:17.012336 6640 raft_consensus.cc:504] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:17.014811 6640 leader_election.cc:288] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:17.016992 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9268f5f6dee4112932e106010a3c2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.018761 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 127e8d0323954ef788018bcb91582951
I20260808 13:35:17.026445 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet 127e8d0323954ef788018bcb91582951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.027524 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet 127e8d0323954ef788018bcb91582951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:17.028394 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d8d5a3ca724141b58e4e2cea23be68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.034644 6635 tablet.cc:1722] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:17.051638 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet fc9b2e69fe27431dac9eb5ecb9e3e23e
I20260808 13:35:17.057188 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:17.058182 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:17.060105 6033 tablet_bootstrap.cc:492] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:17.060581 6033 ts_tablet_manager.cc:1397] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.288s user 0.010s sys 0.003s
I20260808 13:35:17.037144 6648 raft_consensus.cc:2781] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
W20260808 13:35:17.046756 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 76524.4us
I20260808 13:35:17.055584 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.024209 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 127e8d0323954ef788018bcb91582951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:17.079134 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_3728a7744c9c45beb1755624a26cd410"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:17.080634 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3728a7744c9c45beb1755624a26cd410 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:17.085345 4840 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c5103bab29d549298c0b01fc0a045e7a" } new_table_name: "default.Table_13bc8e545bba490b8fef6864a18684a0" modify_external_catalogs: true
W20260808 13:35:17.091084 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 154356us
I20260808 13:35:17.094033 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "543debff6da54d26bf62260bf8721ff7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:17.094723 6402 raft_consensus.cc:482] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:17.094696 4823 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:37135, user_credentials={real_user=slave}} blocked reactor thread for 67618us
I20260808 13:35:17.095438 6402 raft_consensus.cc:504] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:17.098001 6633 log.cc:924] Time spent T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
I20260808 13:35:17.098249 6402 leader_election.cc:288] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:17.109088 5386 raft_consensus.cc:3037] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:17.105262 6026 tablet_bootstrap.cc:654] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:17.115372 5384 raft_consensus.cc:3037] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:17.074210 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 252746us
I20260808 13:35:17.125415 6645 tablet.cc:1722] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:17.139866 6046 tablet_bootstrap.cc:492] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:17.140303 6046 ts_tablet_manager.cc:1397] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.194s user 0.009s sys 0.005s
I20260808 13:35:17.154438 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet a0b29786cb404bc2b01473c3b8390a50
I20260808 13:35:17.156088 4842 catalog_manager.cc:6565] ReplaceTablet: tablet a0b29786cb404bc2b01473c3b8390a50 deleted and replaced by tablet 362dbd6990e740b39690537af98aa2f8
I20260808 13:35:17.156534 6591 ts_tablet_manager.cc:1918] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:17.159121 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d13cbde6844079c2d1eb80732bb1f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:17.167115 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b29786cb404bc2b01473c3b8390a50 with delete_type TABLET_DATA_DELETED (replaced by tablet 362dbd6990e740b39690537af98aa2f8) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:17.169349 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b29786cb404bc2b01473c3b8390a50 with delete_type TABLET_DATA_DELETED (replaced by tablet 362dbd6990e740b39690537af98aa2f8) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:17.170147 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_854eb2acc74c43a688e89cd2fb1f3b96"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:17.172058 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_854eb2acc74c43a688e89cd2fb1f3b96 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:17.174953 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c3535f396f74749a6a28907c6d93abd" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:17.176060 5881 raft_consensus.cc:2454] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
W20260808 13:35:17.189396 5285 leader_election.cc:341] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: Tablet not RUNNING: STOPPED
I20260808 13:35:17.194195 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c3535f396f74749a6a28907c6d93abd" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:17.195313 5383 raft_consensus.cc:2454] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:17.196734 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b29786cb404bc2b01473c3b8390a50 with delete_type TABLET_DATA_DELETED (replaced by tablet 362dbd6990e740b39690537af98aa2f8) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.199554 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet e2a95712de254c13b3b6dc4fe5cb15f7
I20260808 13:35:17.201459 4838 catalog_manager.cc:6565] ReplaceTablet: tablet e2a95712de254c13b3b6dc4fe5cb15f7 deleted and replaced by tablet 3a27517560d64a3b8da2cf55980e18a8
I20260808 13:35:17.205405 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]). Expected version 1 got 0
I20260808 13:35:17.202390 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "543debff6da54d26bf62260bf8721ff7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:17.211033 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a95712de254c13b3b6dc4fe5cb15f7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a27517560d64a3b8da2cf55980e18a8) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:17.214766 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 2 got 0
I20260808 13:35:17.215790 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:17.216300 6046 raft_consensus.cc:348] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:17.217023 6046 raft_consensus.cc:374] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:17.217267 6046 raft_consensus.cc:729] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:17.217782 6046 consensus_queue.cc:260] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:17.223374 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet f0c76b819474468baac51365360becfc (table default.Table_9b3bbebdc1274b238c30a45d36cf86f7 [id=26fb789fff434bb0825daf68d5378013], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet f0c76b819474468baac51365360becfc consensus state has no leader
I20260808 13:35:17.228591 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a95712de254c13b3b6dc4fe5cb15f7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a27517560d64a3b8da2cf55980e18a8) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:17.252787 4839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_8523d4a015194c64b22f66a22b0dcc06" } modify_external_catalogs: true
I20260808 13:35:17.253620 4839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_8523d4a015194c64b22f66a22b0dcc06" } modify_external_catalogs: true
I20260808 13:35:17.256078 5551 leader_election.cc:302] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:17.258240 4837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_76c233d2deef48ec8ec2fba9d29f6ecb" } modify_external_catalogs: true
I20260808 13:35:17.258854 4837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_76c233d2deef48ec8ec2fba9d29f6ecb" } modify_external_catalogs: true
I20260808 13:35:17.268203 6046 ts_tablet_manager.cc:1428] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.128s user 0.008s sys 0.003s
I20260808 13:35:16.921782 6587 ts_tablet_manager.cc:1918] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:17.271193 6587 log.cc:1196] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/edfa66acceea4b54911d78ee16609ef7
I20260808 13:35:17.272835 6587 ts_tablet_manager.cc:1939] T edfa66acceea4b54911d78ee16609ef7 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
W20260808 13:35:17.273864 5772 consensus_peers.cc:487] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:17.274696 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 95402.5us
W20260808 13:35:17.276077 5772 leader_election.cc:341] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 78d492406c8a4c4186131e02a1559cb4
I20260808 13:35:17.291040 6615 raft_consensus.cc:2781] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:17.291440 6615 raft_consensus.cc:482] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:17.291781 6615 raft_consensus.cc:3037] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:17.313625 6046 tablet_bootstrap.cc:492] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:17.316793 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_b2521a9f129840958fb2b9097b7cf37b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:17.318457 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2521a9f129840958fb2b9097b7cf37b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:17.326370 6046 tablet_bootstrap.cc:654] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:17.178139 6591 log.cc:1196] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/345d13cbde6844079c2d1eb80732bb1f
I20260808 13:35:17.335510 6591 ts_tablet_manager.cc:1939] T 345d13cbde6844079c2d1eb80732bb1f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
W20260808 13:35:17.354022 4840 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c3535f396f74749a6a28907c6d93abd consensus state has no leader
I20260808 13:35:17.356431 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 111) took 1634 ms. Trace:
I20260808 13:35:17.356765 6591 rpcz_store.cc:276] 0808 13:35:15.722329 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:15.722475 (+ 146us) service_pool.cc:227] Handling call
0808 13:35:16.338538 (+616063us) ts_tablet_manager.cc:1145] Deleting tablet 345d13cbde6844079c2d1eb80732bb1f
0808 13:35:16.338598 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:16.440613 (+102015us) spinlock_profiling.cc:243] Waited 73.7 ms on lock 0x7b54000b3bd4. stack: 00007f5a54110033 00007f5a5410ff30 00007f5a53c91534 00000000004db2b4 00000000004db249 00007f5a5b01c471 00007f5a5b09f352 00007f5a5b4b526c 00007f5a5c0c2cfa 00007f5a5c0f2e10 00007f5a5c0d9e0d 00007f5a5c0d9db9 00007f5a5c0d9d49 00007f5a5c0d9d11 00007f5a5c0d8f3d 0000000000580784
0808 13:35:16.730491 (+289878us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:17.155516 (+425025us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:17.356342 (+200826us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":169617792,"tablet-delete.queue_time_us":615432}
I20260808 13:35:17.361073 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 115) took 1002 ms. Trace:
I20260808 13:35:17.361408 6591 rpcz_store.cc:276] 0808 13:35:16.358902 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.359103 (+ 201us) service_pool.cc:227] Handling call
0808 13:35:17.360612 (+1001509us) ts_tablet_manager.cc:1145] Deleting tablet 6826e4e4db2043b3a4fe9e7a9359cab6
0808 13:35:17.360686 (+ 74us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:17.361006 (+ 320us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1000553}
W20260808 13:35:17.375792 5551 leader_election.cc:341] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:17.376947 6641 consensus_queue.cc:1035] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:35:17.380442 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 104490us
I20260808 13:35:17.381505 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 345d13cbde6844079c2d1eb80732bb1f (table default.Table_7ad3b44b6be7476881db975f93533b24 [id=cc59d440766d4d2c983f64bca86e7d21]) successfully deleted
W20260808 13:35:17.382275 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6826e4e4db2043b3a4fe9e7a9359cab6 already in progress: creating tablet
I20260808 13:35:17.391163 6591 tablet_replica.cc:330] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:17.392335 6591 raft_consensus.cc:2229] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:17.393304 6591 raft_consensus.cc:2258] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:17.396260 6591 ts_tablet_manager.cc:1905] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:17.288273 6626 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:54830 (request call id 106) took 1257 ms. Trace:
W20260808 13:35:17.289000 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 90864.3us
W20260808 13:35:17.433130 5842 consensus_meta.cc:287] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.526s user 0.009s sys 0.001s
I20260808 13:35:17.290890 6033 raft_consensus.cc:348] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:17.433866 6033 raft_consensus.cc:374] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:17.434177 6033 raft_consensus.cc:729] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:17.437851 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_2637745cbe454854a74c76f3165627c6" } new_table_name: "default.Table_e38e30a466f94219836c80d87f91c8a9" modify_external_catalogs: true
I20260808 13:35:17.440789 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a95712de254c13b3b6dc4fe5cb15f7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a27517560d64a3b8da2cf55980e18a8) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.446563 6033 consensus_queue.cc:260] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:17.462630 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet e00f2e0b98af431d99894e8778257bc4
I20260808 13:35:17.492277 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f2e0b98af431d99894e8778257bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:17.413352 6626 rpcz_store.cc:276] 0808 13:35:16.031029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.031171 (+ 142us) service_pool.cc:227] Handling call
0808 13:35:17.288132 (+1256961us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:16.032904 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:16.033623 (+ 719us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:16.033674 (+ 51us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:16.033889 (+ 215us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:16.034044 (+ 155us) alter_schema_op.cc:126] START. Timestamp: 1786196116033775
0808 13:35:16.034085 (+ 41us) op_driver.cc:331] REPLICATION: starting
0808 13:35:16.035136 (+ 1051us) log.cc:842] Serialized 173 byte log entry
0808 13:35:16.638158 (+603022us) op_driver.cc:444] REPLICATION: finished
0808 13:35:16.804082 (+165924us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:16.806630 (+ 2548us) log.cc:842] Serialized 5 byte log entry
0808 13:35:17.077032 (+270402us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:17.286926 (+209894us) log.cc:842] Serialized 14 byte log entry
0808 13:35:17.287807 (+ 881us) alter_schema_op.cc:71] Released schema lock
0808 13:35:17.287870 (+ 63us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":165692,"prepare.queue_time_us":169,"prepare.run_cpu_time_us":13314,"prepare.run_wall_time_us":44486,"raft.queue_time_us":2810,"raft.run_cpu_time_us":2906,"raft.run_wall_time_us":4194,"replication_time_us":603940,"thread_start_us":260355,"threads_started":3,"wal-append.queue_time_us":32177}]]}
I20260808 13:35:17.502856 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f2e0b98af431d99894e8778257bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.525496 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6ba381f21fc24362bc6d18499af1d329"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:17.527220 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ba381f21fc24362bc6d18499af1d329 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:17.537322 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8ba098c946794607b2a15bc64098d918 consensus state has no leader
I20260808 13:35:17.559739 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fe0257b298fc4b1b933e26b3d8d747bd" } new_table_name: "default.Table_7353a29b611e4f519b3e47a276cd7b5a" modify_external_catalogs: true
W20260808 13:35:17.574091 6678 log.cc:924] Time spent T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: Append to log took a long time: real 0.109s user 0.001s sys 0.000s
I20260808 13:35:17.580585 5834 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 121) took 1010 ms. Trace:
I20260808 13:35:17.581174 5834 rpcz_store.cc:276] 0808 13:35:16.570873 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.594073 (+ 23200us) service_pool.cc:227] Handling call
0808 13:35:16.667714 (+ 73641us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:16.667799 (+ 85us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:16.871188 (+203389us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:17.372593 (+501405us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:17.580507 (+207914us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":207270}
I20260808 13:35:17.598348 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_f685324c68e647af9888454398dc7d23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:17.599908 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f685324c68e647af9888454398dc7d23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:17.408481 6615 raft_consensus.cc:504] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:17.538793 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f2e0b98af431d99894e8778257bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:17.568042 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 116) took 1144 ms. Trace:
I20260808 13:35:17.614560 6615 leader_election.cc:288] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:17.625190 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d (DEFAULT_TABLE table=default.Table_d70de767ab2548b4a9e45418eefc9a2b [id=10bb97e006904dc9b34ebedad3ec13de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.627188 5609 tablet_service.cc:1467] Processing CreateTablet for tablet 275b6aeaad044ea1a16ecd82f63dc59f (DEFAULT_TABLE table=default.Table_4ce2d44dea88418eb0e636202d9df995 [id=31d10fd1087f45349072cd842a68c4a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.629798 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 369a942248bd4cec9ee9170fe55601a3 (DEFAULT_TABLE table=default.Table_7d31aaee9dd04cabbbbca675bde3a2f4 [id=e9b13612cdc742eb88f83d7ce4e999fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.631383 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369a942248bd4cec9ee9170fe55601a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.636037 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 369a942248bd4cec9ee9170fe55601a3 (DEFAULT_TABLE table=default.Table_7d31aaee9dd04cabbbbca675bde3a2f4 [id=e9b13612cdc742eb88f83d7ce4e999fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.636780 5841 tablet_service.cc:1467] Processing CreateTablet for tablet d8b07b7d31ef41e484389c0f007598f0 (DEFAULT_TABLE table=default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.637822 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b07b7d31ef41e484389c0f007598f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.638468 5610 tablet_service.cc:1467] Processing CreateTablet for tablet d8b07b7d31ef41e484389c0f007598f0 (DEFAULT_TABLE table=default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:17.605295 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 346001us
I20260808 13:35:17.643383 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d (DEFAULT_TABLE table=default.Table_d70de767ab2548b4a9e45418eefc9a2b [id=10bb97e006904dc9b34ebedad3ec13de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.643914 5603 tablet_service.cc:1467] Processing CreateTablet for tablet adf9a03742a14cfe867c578c466838de (DEFAULT_TABLE table=default.Table_d6f23c83ff0a4358a7f096203284c706 [id=93919cc7207f4ca09453647840c9bed2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.645998 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b57966a9b7b4673ab9f98720cfd456d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.648854 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 72c4ddec62c242c88b3a759d8361cd30 (DEFAULT_TABLE table=default.Table_920e3ee624f44ee4b19c15e0d7fb4358 [id=844dc96c93564895911d827c23c02689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.649901 6026 tablet_bootstrap.cc:492] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:17.651289 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369a942248bd4cec9ee9170fe55601a3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:17.651753 5550 leader_election.cc:341] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:17.652354 5550 leader_election.cc:302] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:17.653605 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b57966a9b7b4673ab9f98720cfd456d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.655140 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b07b7d31ef41e484389c0f007598f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.659704 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adf9a03742a14cfe867c578c466838de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.647176 5835 tablet_service.cc:1467] Processing CreateTablet for tablet adf9a03742a14cfe867c578c466838de (DEFAULT_TABLE table=default.Table_d6f23c83ff0a4358a7f096203284c706 [id=93919cc7207f4ca09453647840c9bed2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.657442 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 72c4ddec62c242c88b3a759d8361cd30 (DEFAULT_TABLE table=default.Table_920e3ee624f44ee4b19c15e0d7fb4358 [id=844dc96c93564895911d827c23c02689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.663123 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72c4ddec62c242c88b3a759d8361cd30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.663487 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72c4ddec62c242c88b3a759d8361cd30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.667548 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 2 got 0
I20260808 13:35:17.671336 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adf9a03742a14cfe867c578c466838de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.671567 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 275b6aeaad044ea1a16ecd82f63dc59f (DEFAULT_TABLE table=default.Table_4ce2d44dea88418eb0e636202d9df995 [id=31d10fd1087f45349072cd842a68c4a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.672814 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 275b6aeaad044ea1a16ecd82f63dc59f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.675432 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d (DEFAULT_TABLE table=default.Table_d70de767ab2548b4a9e45418eefc9a2b [id=10bb97e006904dc9b34ebedad3ec13de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.604187 6587 rpcz_store.cc:276] 0808 13:35:16.434156 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.434285 (+ 129us) service_pool.cc:227] Handling call
0808 13:35:16.437402 (+ 3117us) ts_tablet_manager.cc:1145] Deleting tablet edfa66acceea4b54911d78ee16609ef7
0808 13:35:16.438426 (+ 1024us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:16.710860 (+272434us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:16.921545 (+210685us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:17.564938 (+643393us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2435}
I20260808 13:35:17.677723 5346 tablet_service.cc:1467] Processing CreateTablet for tablet 369a942248bd4cec9ee9170fe55601a3 (DEFAULT_TABLE table=default.Table_7d31aaee9dd04cabbbbca675bde3a2f4 [id=e9b13612cdc742eb88f83d7ce4e999fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.679021 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_fd5435dd94cd493ab4780d4dea9de9a8 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 2 got 1
I20260808 13:35:17.678998 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 119) took 1084 ms. Trace:
I20260808 13:35:17.679601 6587 rpcz_store.cc:276] 0808 13:35:16.595120 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.596886 (+ 1766us) service_pool.cc:227] Handling call
0808 13:35:17.678327 (+1081441us) ts_tablet_manager.cc:1145] Deleting tablet 9260370cf8cb46a7b699581e4e8d46db
0808 13:35:17.678398 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:17.678889 (+ 491us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1076589}
I20260808 13:35:17.681953 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_0ae56405f82f446dbd5ed65b0062fc62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:17.682132 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c3535f396f74749a6a28907c6d93abd" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:17.682984 5386 raft_consensus.cc:3037] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:17.683539 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ae56405f82f446dbd5ed65b0062fc62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:17.685397 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d81d55e146d84d41a68894ad0dd4dc93" } modify_external_catalogs: true
I20260808 13:35:17.685868 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d81d55e146d84d41a68894ad0dd4dc93" } modify_external_catalogs: true
I20260808 13:35:17.687723 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b (DEFAULT_TABLE table=default.Table_f212f0d1d7294b4f80606ba37c7f19f3 [id=8039834d1c7f4cb6894534c8284b27c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.689268 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 196d80d806b0432cacf2ca6aa0da8c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.693768 5845 tablet_service.cc:1467] Processing CreateTablet for tablet f85b4fd30d6c4903946891a399acf7a0 (DEFAULT_TABLE table=default.Table_7da887b39f244cf0b44cbe9fa3b0eae2 [id=ea2ba343c17d403d8cb0bf9d80bee453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.679975 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:17.695094 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f85b4fd30d6c4903946891a399acf7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.699110 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b57966a9b7b4673ab9f98720cfd456d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.708278 6587 tablet_replica.cc:330] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
W20260808 13:35:17.712390 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b8ea98afef6841fead641b314ee53a6e consensus state has no leader
I20260808 13:35:17.694917 4838 catalog_manager.cc:5495] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:17.720733 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
W20260808 13:35:17.727005 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 344963us
I20260808 13:35:17.711452 5354 tablet_service.cc:1467] Processing CreateTablet for tablet adf9a03742a14cfe867c578c466838de (DEFAULT_TABLE table=default.Table_d6f23c83ff0a4358a7f096203284c706 [id=93919cc7207f4ca09453647840c9bed2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.720830 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]) successfully deleted
W20260808 13:35:17.734097 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 9260370cf8cb46a7b699581e4e8d46db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9260370cf8cb46a7b699581e4e8d46db already in progress: creating tablet
W20260808 13:35:17.734730 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 127e8d0323954ef788018bcb91582951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 127e8d0323954ef788018bcb91582951 already in progress: creating tablet
W20260808 13:35:17.735307 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc9b2e69fe27431dac9eb5ecb9e3e23e already in progress: creating tablet
W20260808 13:35:17.735846 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet a0b29786cb404bc2b01473c3b8390a50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0b29786cb404bc2b01473c3b8390a50 already in progress: creating tablet
I20260808 13:35:17.738574 4832 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce] alter complete (version 1)
I20260808 13:35:17.711951 6046 tablet_bootstrap.cc:492] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:17.712611 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 72c4ddec62c242c88b3a759d8361cd30 (DEFAULT_TABLE table=default.Table_920e3ee624f44ee4b19c15e0d7fb4358 [id=844dc96c93564895911d827c23c02689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.714001 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b (DEFAULT_TABLE table=default.Table_f212f0d1d7294b4f80606ba37c7f19f3 [id=8039834d1c7f4cb6894534c8284b27c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.710263 5356 tablet_service.cc:1467] Processing CreateTablet for tablet d8b07b7d31ef41e484389c0f007598f0 (DEFAULT_TABLE table=default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.741130 6046 ts_tablet_manager.cc:1397] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.428s user 0.011s sys 0.009s
W20260808 13:35:17.571239 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 391937us
W20260808 13:35:17.751605 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 40) took 966 ms (client timeout 1000 ms). Trace:
W20260808 13:35:17.752018 4835 rpcz_store.cc:269] 0808 13:35:16.785650 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.785849 (+ 199us) service_pool.cc:227] Handling call
0808 13:35:16.814636 (+ 28787us) spinlock_profiling.cc:243] Waited 8.06 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:17.751543 (+936907us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2703,"reported_tablets":17,"spinlock_wait_cycles":18540800}
W20260808 13:35:17.751605 4841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 39) took 2126 ms (client timeout 1000 ms). Trace:
W20260808 13:35:17.754258 4841 rpcz_store.cc:269] 0808 13:35:15.625693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:15.761691 (+135998us) service_pool.cc:227] Handling call
0808 13:35:17.751540 (+1989849us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:16.745962 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:16.746313 (+ 351us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:16.746335 (+ 22us) write_op.cc:435] Acquired schema lock
0808 13:35:16.746348 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:16.746660 (+ 312us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:16.746699 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:16.746731 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:16.746998 (+ 267us) tablet.cc:699] Row locks acquired
0808 13:35:16.747019 (+ 21us) write_op.cc:260] PREPARE: finished
0808 13:35:16.747127 (+ 108us) write_op.cc:270] Start()
0808 13:35:16.747208 (+ 81us) write_op.cc:276] Timestamp: P: 1786196116747042 usec, L: 0
0808 13:35:16.747223 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:16.755993 (+ 8770us) log.cc:842] Serialized 901 byte log entry
0808 13:35:16.758010 (+ 2017us) spinlock_profiling.cc:243] Waited 8.21 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:16.770929 (+ 12919us) op_driver.cc:444] REPLICATION: finished
0808 13:35:16.774222 (+ 3293us) write_op.cc:301] APPLY: starting
0808 13:35:16.774284 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:16.774404 (+ 120us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:16.774430 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:16.774754 (+ 324us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:16.775005 (+ 251us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:16.775064 (+ 59us) write_op.cc:312] APPLY: finished
0808 13:35:16.775268 (+ 204us) log.cc:842] Serialized 29 byte log entry
0808 13:35:16.776107 (+ 839us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:16.776194 (+ 87us) write_op.cc:454] Released schema lock
0808 13:35:16.776230 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:16.779847 (+ 3617us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:16.779874 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":15,"spinlock_wait_cycles":19712,"child_traces":[["op",{"apply.queue_time_us":3116,"apply.run_cpu_time_us":1762,"apply.run_wall_time_us":5852,"num_ops":2,"prepare.queue_time_us":204231,"prepare.run_cpu_time_us":3034,"prepare.run_wall_time_us":45130,"raft.queue_time_us":152,"raft.run_cpu_time_us":935,"raft.run_wall_time_us":2011,"replication_time_us":23617,"spinlock_wait_cycles":18880256}]]}
W20260808 13:35:17.762598 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 32) took 1175 ms (client timeout 1000 ms). Trace:
I20260808 13:35:17.762152 6626 tablet.cc:1722] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:17.767695 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_206fd86bd0f44e8e9dc38631edff49ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:17.769362 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_206fd86bd0f44e8e9dc38631edff49ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:17.771350 4840 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_595f29cce77d4dbf8e1608b573611ad4" } new_table_name: "default.Table_b938fae0ac404bf6acd3d8bef2b83cfe" modify_external_catalogs: true
I20260808 13:35:17.771521 5345 tablet_service.cc:1514] Processing DeleteTablet for tablet edfa66acceea4b54911d78ee16609ef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:15 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.772897 5351 tablet_service.cc:1467] Processing CreateTablet for tablet f85b4fd30d6c4903946891a399acf7a0 (DEFAULT_TABLE table=default.Table_7da887b39f244cf0b44cbe9fa3b0eae2 [id=ea2ba343c17d403d8cb0bf9d80bee453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.781351 4837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:17.770819 5346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369a942248bd4cec9ee9170fe55601a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.784193 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adf9a03742a14cfe867c578c466838de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.785283 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.787108 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.788384 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d492406c8a4c4186131e02a1559cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.789666 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f85b4fd30d6c4903946891a399acf7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.791106 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 196d80d806b0432cacf2ca6aa0da8c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.793752 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b07b7d31ef41e484389c0f007598f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.794873 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72c4ddec62c242c88b3a759d8361cd30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.795950 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_0af988c3a5a243daa24e072c08164aaa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:17.797977 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0af988c3a5a243daa24e072c08164aaa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:17.801963 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fb3e099f7f4544b3c100070f0d14ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:17.803470 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 52742.7us
I20260808 13:35:17.803916 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d13cbde6844079c2d1eb80732bb1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:17.747185 6648 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.565s user 0.013s sys 0.041s
I20260808 13:35:17.811048 5345 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.815990 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 8948ce98e7a941f59fa9932fe241638d with delete_type TABLET_DATA_DELETED (replaced by tablet d9268f5f6dee4112932e106010a3c2be) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.816470 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet d2ac4c366363468aa62b73e0f1e04532
I20260808 13:35:17.818778 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet d2ac4c366363468aa62b73e0f1e04532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:17.828428 6626 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:54830 (request call id 114) took 1304 ms. Trace:
I20260808 13:35:17.829195 6626 rpcz_store.cc:276] 0808 13:35:16.523636 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.524644 (+ 1008us) service_pool.cc:227] Handling call
0808 13:35:16.581321 (+ 56677us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b5800001898. stack: 00007f19df247033 00007f19df246f30 00007f19dedc8534 00000000004db2b4 00000000004db249 00007f19df0bdf11 00007f19df241b7c 00007f19df24173b 00007f19e80d8b09 00007f19e7135730 00007f19e71fb79b 00007f19e71fb821 00007f19e71ba9d0 00007f19e71a4a39 00007f19e71a4084 00007f19e589d744
0808 13:35:17.828324 (+1247003us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:16.628963 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:16.786155 (+157192us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:16.789971 (+ 3816us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:16.794858 (+ 4887us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:16.794976 (+ 118us) alter_schema_op.cc:126] START. Timestamp: 1786196116782900
0808 13:35:16.794995 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:16.795560 (+ 565us) log.cc:842] Serialized 173 byte log entry
0808 13:35:17.758198 (+962638us) op_driver.cc:444] REPLICATION: finished
0808 13:35:17.762000 (+ 3802us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:17.764092 (+ 2092us) log.cc:842] Serialized 5 byte log entry
0808 13:35:17.813314 (+ 49222us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:17.827973 (+ 14659us) log.cc:842] Serialized 14 byte log entry
0808 13:35:17.828122 (+ 149us) alter_schema_op.cc:71] Released schema lock
0808 13:35:17.828167 (+ 45us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"spinlock_wait_cycles":10812160,"child_traces":[["op",{"apply.queue_time_us":3544,"prepare.queue_time_us":46885,"prepare.run_cpu_time_us":10079,"prepare.run_wall_time_us":410340,"raft.queue_time_us":2983,"raft.run_cpu_time_us":1178,"raft.run_wall_time_us":1228,"replication_time_us":963087,"thread_start_us":319927,"threads_started":2,"wal-append.queue_time_us":219656}]]}
W20260808 13:35:17.762995 4832 rpcz_store.cc:269] 0808 13:35:16.587480 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.591671 (+ 4191us) service_pool.cc:227] Handling call
0808 13:35:16.681912 (+ 90241us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:17.762529 (+1080617us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:17.300588 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:17.302704 (+ 2116us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:17.302740 (+ 36us) write_op.cc:435] Acquired schema lock
0808 13:35:17.302993 (+ 253us) tablet.cc:660] Decoding operations
0808 13:35:17.304342 (+ 1349us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:17.304382 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:17.304402 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:17.304535 (+ 133us) tablet.cc:699] Row locks acquired
0808 13:35:17.304548 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:17.304645 (+ 97us) write_op.cc:270] Start()
0808 13:35:17.304722 (+ 77us) write_op.cc:276] Timestamp: P: 1786196117304604 usec, L: 0
0808 13:35:17.304736 (+ 14us) op_driver.cc:331] REPLICATION: starting
0808 13:35:17.310929 (+ 6193us) log.cc:842] Serialized 356 byte log entry
0808 13:35:17.347498 (+ 36569us) op_driver.cc:444] REPLICATION: finished
0808 13:35:17.350035 (+ 2537us) write_op.cc:301] APPLY: starting
0808 13:35:17.350103 (+ 68us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:17.350230 (+ 127us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:17.350346 (+ 116us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:17.350581 (+ 235us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:17.355122 (+ 4541us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:17.355174 (+ 52us) write_op.cc:312] APPLY: finished
0808 13:35:17.355347 (+ 173us) log.cc:842] Serialized 23 byte log entry
0808 13:35:17.355479 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:17.355542 (+ 63us) write_op.cc:454] Released schema lock
0808 13:35:17.355570 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:17.355748 (+ 178us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:17.355785 (+ 37us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":18,"spinlock_wait_cycles":44002048,"child_traces":[["op",{"apply.queue_time_us":2354,"apply.run_cpu_time_us":1551,"apply.run_wall_time_us":5917,"num_ops":1,"prepare.queue_time_us":270975,"prepare.run_cpu_time_us":2237,"prepare.run_wall_time_us":22615,"raft.queue_time_us":2434,"raft.run_cpu_time_us":930,"raft.run_wall_time_us":1798,"replication_time_us":42661}]]}
I20260808 13:35:17.745208 6653 raft_consensus.cc:482] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:17.840574 6585 ts_tablet_manager.cc:1918] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:17.841078 6585 log.cc:1196] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/edfa66acceea4b54911d78ee16609ef7
I20260808 13:35:17.842072 6585 ts_tablet_manager.cc:1939] T edfa66acceea4b54911d78ee16609ef7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:17.812566 6648 raft_consensus.cc:686] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
W20260808 13:35:17.846047 5771 consensus_peers.cc:487] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:17.863075 6033 ts_tablet_manager.cc:1428] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.802s user 0.002s sys 0.007s
W20260808 13:35:17.863629 6033 ts_tablet_manager.cc:1460] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Tablet startup took 802ms
W20260808 13:35:17.863947 6033 ts_tablet_manager.cc:1462] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Trace:
0808 13:35:09.273158 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:09.273407 (+ 249us) service_pool.cc:227] Handling call
0808 13:35:09.290512 (+ 17105us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:09.290719 (+ 207us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:09.403974 (+113255us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:09.508516 (+104542us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:09.509136 (+ 620us) inbound_call.cc:173] Queueing success response
0808 13:35:16.772816 (+7263680us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0808 13:35:17.061025 (+288209us) ts_tablet_manager.cc:1429] Starting tablet replica
0808 13:35:17.062206 (+ 1181us) tablet_replica.cc:259] Starting instrumentation
0808 13:35:17.216815 (+154609us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":72857,"raft.queue_time_us":848,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":1002,"tablet-open.queue_time_us":7263813}
I20260808 13:35:17.866431 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 116) took 1342 ms. Trace:
I20260808 13:35:17.867050 6585 rpcz_store.cc:276] 0808 13:35:16.524494 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.524645 (+ 151us) service_pool.cc:227] Handling call
0808 13:35:16.835324 (+310679us) ts_tablet_manager.cc:1145] Deleting tablet edfa66acceea4b54911d78ee16609ef7
0808 13:35:16.835359 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:17.418488 (+583129us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:17.840250 (+421762us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:17.866356 (+ 26106us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17698,"tablet-delete.queue_time_us":310057}
I20260808 13:35:17.868592 6585 tablet_replica.cc:330] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:17.866778 5834 tablet_service.cc:1467] Processing CreateTablet for tablet 275b6aeaad044ea1a16ecd82f63dc59f (DEFAULT_TABLE table=default.Table_4ce2d44dea88418eb0e636202d9df995 [id=31d10fd1087f45349072cd842a68c4a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:17.870705 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]) successfully deleted
I20260808 13:35:17.870924 6585 raft_consensus.cc:2229] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 LEADER]: Raft consensus shutting down.
I20260808 13:35:17.871300 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 275b6aeaad044ea1a16ecd82f63dc59f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:17.874575 6585 pending_rounds.cc:62] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Trying to abort 1 pending ops.
I20260808 13:35:17.874979 6585 pending_rounds.cc:68] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Aborting op as it isn't in flight: id { term: 2 index: 1 } timestamp: 7316259295018721280 op_type: NO_OP noop_request { }
I20260808 13:35:17.885866 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet d2ac4c366363468aa62b73e0f1e04532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.875586 6585 raft_consensus.cc:2866] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:35:17.895268 6585 raft_consensus.cc:2258] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:17.902112 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fb3e099f7f4544b3c100070f0d14ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:17.904125 6587 raft_consensus.cc:2229] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:17.904558 6585 ts_tablet_manager.cc:1905] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:17.904662 6587 pending_rounds.cc:62] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: Trying to abort 1 pending ops.
I20260808 13:35:17.904919 6587 pending_rounds.cc:68] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316259294342758400 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "e2a95712de254c13b3b6dc4fe5cb15f7" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_0a3697cdad3740a1baa47b3075b5e1f9" dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" new_extra_config { } }
I20260808 13:35:17.884646 6653 raft_consensus.cc:504] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:17.939788 6046 raft_consensus.cc:348] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:17.940459 4836 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_cba3fa83e59546fd8d7817c6ba750080" } new_table_name: "default.Table_5c71e9adaaf5477b996f88813566c561" modify_external_catalogs: true
I20260808 13:35:17.944931 6046 raft_consensus.cc:374] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:17.945899 6046 raft_consensus.cc:729] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:35:17.962190 4840 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c4c083c8e19d4279af600670f5d1d78d consensus state has no leader
I20260808 13:35:17.970685 6626 tablet.cc:1722] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:17.947188 6046 consensus_queue.cc:260] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:17.906570 6587 raft_consensus.cc:2258] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:17.943557 5283 leader_election.cc:302] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:17.844761 6648 consensus_queue.cc:237] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:17.987176 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d70de767ab2548b4a9e45418eefc9a2b" } modify_external_catalogs: true
I20260808 13:35:17.987814 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d70de767ab2548b4a9e45418eefc9a2b" } modify_external_catalogs: true
I20260808 13:35:18.002360 6677 raft_consensus.cc:2726] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:18.003777 6635 tablet.cc:1722] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:18.010386 6587 ts_tablet_manager.cc:1905] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:18.010754 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 67289.4us
I20260808 13:35:18.022881 6585 ts_tablet_manager.cc:1918] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260808 13:35:18.024091 6585 log.cc:1196] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/78d492406c8a4c4186131e02a1559cb4
I20260808 13:35:18.025008 6585 ts_tablet_manager.cc:1939] T 78d492406c8a4c4186131e02a1559cb4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:18.027053 6653 leader_election.cc:288] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:18.028966 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc9a6ec9a86431fb46e50fcaa658d33" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:18.031149 6046 ts_tablet_manager.cc:1428] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.289s user 0.007s sys 0.000s
I20260808 13:35:18.031781 4838 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8] alter complete (version 1)
W20260808 13:35:18.032837 4823 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:37135, user_credentials={real_user=slave}} blocked reactor thread for 55392.6us
I20260808 13:35:18.033437 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 118) took 1508 ms. Trace:
I20260808 13:35:18.034672 6585 rpcz_store.cc:276] 0808 13:35:16.525357 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.525478 (+ 121us) service_pool.cc:227] Handling call
0808 13:35:17.868401 (+1342923us) ts_tablet_manager.cc:1145] Deleting tablet 78d492406c8a4c4186131e02a1559cb4
0808 13:35:17.868454 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:17.969201 (+100747us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:18.005871 (+ 36670us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:18.033368 (+ 27497us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2600,"tablet-delete.queue_time_us":1342288}
I20260808 13:35:18.036489 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 78d492406c8a4c4186131e02a1559cb4 (table default.Table_6cb245b3d74d4347843035902b8d49d1 [id=c0806126263c41b68137cad5e21de125]) successfully deleted
I20260808 13:35:18.037742 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 122) took 1440 ms. Trace:
I20260808 13:35:18.039224 6585 rpcz_store.cc:276] 0808 13:35:16.597555 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.597836 (+ 281us) service_pool.cc:227] Handling call
0808 13:35:18.037337 (+1439501us) ts_tablet_manager.cc:1145] Deleting tablet 9260370cf8cb46a7b699581e4e8d46db
0808 13:35:18.037402 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:18.037687 (+ 285us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1438826}
W20260808 13:35:18.042687 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 9260370cf8cb46a7b699581e4e8d46db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9260370cf8cb46a7b699581e4e8d46db already in progress: creating tablet
I20260808 13:35:18.044132 6585 tablet_replica.cc:330] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:18.045485 6585 raft_consensus.cc:2229] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:18.047466 6585 raft_consensus.cc:2258] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:18.062626 6046 tablet_bootstrap.cc:492] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:18.065799 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d492406c8a4c4186131e02a1559cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:18.067549 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet edfa66acceea4b54911d78ee16609ef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:15 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:18.067590 4838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 23) took 822 ms (client timeout 1000 ms). Trace:
I20260808 13:35:17.650535 6026 ts_tablet_manager.cc:1397] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.900s user 0.020s sys 0.005s
I20260808 13:35:18.074327 6615 raft_consensus.cc:2726] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:18.088639 6585 ts_tablet_manager.cc:1905] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:18.090276 6033 tablet_bootstrap.cc:492] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:18.097014 5607 tablet_service.cc:1467] Processing CreateTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b (DEFAULT_TABLE table=default.Table_f212f0d1d7294b4f80606ba37c7f19f3 [id=8039834d1c7f4cb6894534c8284b27c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.099258 5616 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d13cbde6844079c2d1eb80732bb1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:18.100709 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fb3e099f7f4544b3c100070f0d14ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:18.102337 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet d2ac4c366363468aa62b73e0f1e04532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:18.083777 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 275b6aeaad044ea1a16ecd82f63dc59f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:18.083004 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 431307us
I20260808 13:35:18.131794 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 196d80d806b0432cacf2ca6aa0da8c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.038417 5386 raft_consensus.cc:2454] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:18.143360 6653 raft_consensus.cc:482] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:18.144335 6653 raft_consensus.cc:504] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.154572 5611 tablet_service.cc:1467] Processing CreateTablet for tablet f85b4fd30d6c4903946891a399acf7a0 (DEFAULT_TABLE table=default.Table_7da887b39f244cf0b44cbe9fa3b0eae2 [id=ea2ba343c17d403d8cb0bf9d80bee453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.150508 5551 leader_election.cc:302] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:18.173671 6615 raft_consensus.cc:2781] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:18.174180 6615 raft_consensus.cc:686] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:18.175091 6615 consensus_queue.cc:237] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.179291 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f85b4fd30d6c4903946891a399acf7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.184515 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc9a6ec9a86431fb46e50fcaa658d33" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:18.204147 6026 raft_consensus.cc:348] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.204821 6026 raft_consensus.cc:374] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:18.205088 6026 raft_consensus.cc:729] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:18.205639 6026 consensus_queue.cc:260] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.225375 5386 raft_consensus.cc:3037] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:18.228513 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c3535f396f74749a6a28907c6d93abd" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:18.229099 5879 raft_consensus.cc:3037] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:18.236554 6026 ts_tablet_manager.cc:1428] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.164s user 0.006s sys 0.001s
I20260808 13:35:18.237746 6026 tablet_bootstrap.cc:492] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:18.244956 6705 tablet.cc:1722] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260808 13:35:18.274251 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 64629us
I20260808 13:35:18.276983 5285 leader_election.cc:302] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:18.223866 6653 leader_election.cc:288] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:18.280046 4836 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 973a7afdcbf944aab8f11b3fd3995865 consensus state has no leader
I20260808 13:35:18.284660 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_93429697aa7c4405b439864c62c45c29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:18.287405 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_43c8f783f487473eb3ea05192f470ef8" } modify_external_catalogs: true
I20260808 13:35:18.287945 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_43c8f783f487473eb3ea05192f470ef8" } modify_external_catalogs: true
I20260808 13:35:18.289921 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_b9063cf92547431f9c63a652c79fbaae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:18.291682 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9063cf92547431f9c63a652c79fbaae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:18.294112 4832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_38684ebd5f804f82a2132c69d077edfe" } new_table_name: "default.Table_41272f24d0a5431b9f718128ad86cb81" modify_external_catalogs: true
I20260808 13:35:18.294659 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c7d6c5128772436194a73fb2f48105b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:18.295989 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7d6c5128772436194a73fb2f48105b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:18.297605 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_2df92438979447cc89b7a5301a13d6a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:18.298995 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2df92438979447cc89b7a5301a13d6a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:18.286535 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93429697aa7c4405b439864c62c45c29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:18.346669 4821 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 973a7afdcbf944aab8f11b3fd3995865 (table default.Table_5c71e9adaaf5477b996f88813566c561 [id=eb20921cee2f48dc86300ce8ce5f52a1], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 973a7afdcbf944aab8f11b3fd3995865 consensus state has no leader
W20260808 13:35:18.399359 4832 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 896389941179456d9d0ed3cd14726f11 consensus state has no leader
W20260808 13:35:18.068017 4838 rpcz_store.cc:269] 0808 13:35:17.244992 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.245166 (+ 174us) service_pool.cc:227] Handling call
0808 13:35:17.259047 (+ 13881us) spinlock_profiling.cc:243] Waited 8.78 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:18.067506 (+808459us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:17.890019 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:17.890416 (+ 397us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:17.890439 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:17.890454 (+ 15us) tablet.cc:660] Decoding operations
0808 13:35:17.890744 (+ 290us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:17.890787 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:17.890811 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:17.890955 (+ 144us) tablet.cc:699] Row locks acquired
0808 13:35:17.890969 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:17.891072 (+ 103us) write_op.cc:270] Start()
0808 13:35:17.891153 (+ 81us) write_op.cc:276] Timestamp: P: 1786196117890790 usec, L: 0
0808 13:35:17.891170 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:17.891653 (+ 483us) log.cc:842] Serialized 527 byte log entry
0808 13:35:17.908402 (+ 16749us) op_driver.cc:444] REPLICATION: finished
0808 13:35:17.908767 (+ 365us) write_op.cc:301] APPLY: starting
0808 13:35:17.908827 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:17.908922 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:17.909037 (+ 115us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:17.909198 (+ 161us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:17.909345 (+ 147us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:17.909392 (+ 47us) write_op.cc:312] APPLY: finished
0808 13:35:17.909580 (+ 188us) log.cc:842] Serialized 23 byte log entry
0808 13:35:17.909722 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:17.909790 (+ 68us) write_op.cc:454] Released schema lock
0808 13:35:17.909823 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:17.910003 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:17.910028 (+ 25us) write_op.cc:454] Released schema lock
Related trace 'op':
0808 13:35:17.979950 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:17.980308 (+ 358us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:17.980331 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:17.980344 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:17.980591 (+ 247us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:17.980625 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:17.980647 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:17.980795 (+ 148us) tablet.cc:699] Row locks acquired
0808 13:35:17.980810 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:17.980903 (+ 93us) write_op.cc:270] Start()
0808 13:35:17.980978 (+ 75us) write_op.cc:276] Timestamp: P: 1786196117980615 usec, L: 0
0808 13:35:17.980994 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:17.981437 (+ 443us) log.cc:842] Serialized 356 byte log entry
0808 13:35:18.011499 (+ 30062us) op_driver.cc:444] REPLICATION: finished
0808 13:35:18.025199 (+ 13700us) write_op.cc:301] APPLY: starting
0808 13:35:18.025424 (+ 225us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:18.025551 (+ 127us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:18.025673 (+ 122us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:18.025876 (+ 203us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:18.026072 (+ 196us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:18.026138 (+ 66us) write_op.cc:312] APPLY: finished
0808 13:35:18.029708 (+ 3570us) log.cc:842] Serialized 23 byte log entry
0808 13:35:18.030790 (+ 1082us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:18.030876 (+ 86us) write_op.cc:454] Released schema lock
0808 13:35:18.030921 (+ 45us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:18.060185 (+ 29264us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:18.060243 (+ 58us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":19,"spinlock_wait_cycles":20187904,"child_traces":[["op",{"apply.queue_time_us":219,"apply.run_cpu_time_us":1423,"apply.run_wall_time_us":1423,"num_ops":1,"prepare.queue_time_us":123324,"prepare.run_cpu_time_us":1998,"prepare.run_wall_time_us":45546,"raft.queue_time_us":281,"raft.run_cpu_time_us":720,"raft.run_wall_time_us":1551,"replication_time_us":17155}],["op",{"apply.queue_time_us":13550,"apply.run_cpu_time_us":2096,"apply.run_wall_time_us":35263,"num_ops":1,"prepare.queue_time_us":55535,"prepare.run_cpu_time_us":1672,"prepare.run_wall_time_us":1672,"replication_time_us":30434,"thread_start_us":265480,"threads_started":1}]]}
I20260808 13:35:18.401939 6033 tablet_bootstrap.cc:654] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:18.400946 6634 raft_consensus.cc:482] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5cd987c3dbdb4168beb73cd7e577c7cb)
I20260808 13:35:18.403295 6634 raft_consensus.cc:504] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.410792 6485 raft_consensus.cc:482] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:18.411335 6485 raft_consensus.cc:504] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.411902 6634 leader_election.cc:288] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:18.414933 6485 leader_election.cc:288] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:18.417366 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 267660us
I20260808 13:35:18.447366 6485 consensus_queue.cc:1035] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:18.455224 6046 tablet_bootstrap.cc:654] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:18.459769 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "543debff6da54d26bf62260bf8721ff7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:18.464464 5771 leader_election.cc:341] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:18.466768 6680 raft_consensus.cc:2726] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:18.469496 6585 ts_tablet_manager.cc:1918] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:18.470217 6585 log.cc:1196] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/345d13cbde6844079c2d1eb80732bb1f
I20260808 13:35:18.484640 6585 ts_tablet_manager.cc:1939] T 345d13cbde6844079c2d1eb80732bb1f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:18.486034 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 123) took 1867 ms. Trace:
I20260808 13:35:18.513965 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 851a0b374a854efd87dd66e39b9df2d5 (DEFAULT_TABLE table=default.Table_6ba381f21fc24362bc6d18499af1d329 [id=393e6ebfd240438297030391565be087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.515307 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 851a0b374a854efd87dd66e39b9df2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.519551 5384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.170.2:56477 (request call id 63) took 1411 ms. Trace:
I20260808 13:35:18.472071 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344828b16cfc47d5b419f8b569055fa4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:18.535706 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 3b1a27f687b54bc592ef4efb72b7247f (DEFAULT_TABLE table=default.Table_661e11184b9346b1bbae005034dddd62 [id=0ded1d2205b248c8acf4ba8baac43096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.527467 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8cf06462484c99a860762a82f7ad64 (DEFAULT_TABLE table=default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.543440 4839 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_2dfd25125045402e8caf41f11924eb9a" } new_table_name: "default.Table_d9c69f374e33451f843637e7cf776be0" modify_external_catalogs: true
I20260808 13:35:18.545107 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 023fe27fc8ce4321a43a1f6037dd9cce (DEFAULT_TABLE table=default.Table_854eb2acc74c43a688e89cd2fb1f3b96 [id=65c22acaec6e40f78aa828752330ed2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.546291 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 3b1a27f687b54bc592ef4efb72b7247f (DEFAULT_TABLE table=default.Table_661e11184b9346b1bbae005034dddd62 [id=0ded1d2205b248c8acf4ba8baac43096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.547515 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b1a27f687b54bc592ef4efb72b7247f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.567116 5635 raft_consensus.cc:2379] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:18.573374 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6e6ce417cc45fd90f5858c30d37a47" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:18.520103 5384 rpcz_store.cc:276] 0808 13:35:17.108521 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.114729 (+ 6208us) service_pool.cc:227] Handling call
0808 13:35:17.115197 (+ 468us) raft_consensus.cc:1457] Updating replica for 1 ops
0808 13:35:17.363063 (+247866us) raft_consensus.cc:1520] Early marking committed up to index 0
0808 13:35:17.407709 (+ 44646us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0808 13:35:17.553823 (+146114us) log.cc:842] Serialized 26 byte log entry
0808 13:35:18.040090 (+486267us) raft_consensus.cc:1631] Marking committed up to 1
0808 13:35:18.241090 (+201000us) log.cc:842] Serialized 14 byte log entry
0808 13:35:18.390010 (+148920us) raft_consensus.cc:1683] Filling consensus response to leader.
0808 13:35:18.398432 (+ 8422us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0808 13:35:18.519248 (+120816us) raft_consensus.cc:1671] finished
0808 13:35:18.519272 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0808 13:35:18.519485 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":37453,"prepare.queue_time_us":171752,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":201432,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":373,"thread_start_us":340550,"threads_started":1,"wal-append.queue_time_us":665011}
I20260808 13:35:18.577342 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 851a0b374a854efd87dd66e39b9df2d5 (DEFAULT_TABLE table=default.Table_6ba381f21fc24362bc6d18499af1d329 [id=393e6ebfd240438297030391565be087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.579540 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 851a0b374a854efd87dd66e39b9df2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.588485 5382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.170.2:56477 (request call id 66) took 1164 ms. Trace:
I20260808 13:35:18.590920 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 264ee45f8dfa4a05b0e5e812ce1b622c
I20260808 13:35:18.591775 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "543debff6da54d26bf62260bf8721ff7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:18.592530 5636 raft_consensus.cc:2454] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:18.599265 6026 tablet_bootstrap.cc:654] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:18.581005 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8cf06462484c99a860762a82f7ad64 (DEFAULT_TABLE table=default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.600129 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:18.608682 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 8b57966a9b7b4673ab9f98720cfd456d
I20260808 13:35:18.510094 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8cf06462484c99a860762a82f7ad64 (DEFAULT_TABLE table=default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.593892 5772 leader_election.cc:302] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:18.618687 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8cf06462484c99a860762a82f7ad64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.619094 6328 raft_consensus.cc:2781] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:18.619518 6328 raft_consensus.cc:482] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:18.619869 6328 raft_consensus.cc:3037] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:18.614703 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:18.532394 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 851a0b374a854efd87dd66e39b9df2d5 (DEFAULT_TABLE table=default.Table_6ba381f21fc24362bc6d18499af1d329 [id=393e6ebfd240438297030391565be087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.623576 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 851a0b374a854efd87dd66e39b9df2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.493381 6604 consensus_queue.cc:1035] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.134s
I20260808 13:35:18.486589 6585 rpcz_store.cc:276] 0808 13:35:16.618889 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.619144 (+ 255us) service_pool.cc:227] Handling call
0808 13:35:18.043632 (+1424488us) ts_tablet_manager.cc:1145] Deleting tablet 345d13cbde6844079c2d1eb80732bb1f
0808 13:35:18.044009 (+ 377us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:18.226935 (+182926us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:18.469122 (+242187us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:18.485924 (+ 16802us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1423448}
I20260808 13:35:18.628082 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 345d13cbde6844079c2d1eb80732bb1f (table default.Table_7ad3b44b6be7476881db975f93533b24 [id=cc59d440766d4d2c983f64bca86e7d21]) successfully deleted
I20260808 13:35:18.628294 6328 raft_consensus.cc:504] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.628976 6585 tablet_replica.cc:330] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:18.596520 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:18.512529 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 023fe27fc8ce4321a43a1f6037dd9cce (DEFAULT_TABLE table=default.Table_854eb2acc74c43a688e89cd2fb1f3b96 [id=65c22acaec6e40f78aa828752330ed2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.633129 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 023fe27fc8ce4321a43a1f6037dd9cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.623999 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 023fe27fc8ce4321a43a1f6037dd9cce (DEFAULT_TABLE table=default.Table_854eb2acc74c43a688e89cd2fb1f3b96 [id=65c22acaec6e40f78aa828752330ed2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.636857 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 3b1a27f687b54bc592ef4efb72b7247f (DEFAULT_TABLE table=default.Table_661e11184b9346b1bbae005034dddd62 [id=0ded1d2205b248c8acf4ba8baac43096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.647383 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 023fe27fc8ce4321a43a1f6037dd9cce. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:18.616369 5642 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s user 0.014s sys 0.041s
I20260808 13:35:18.651666 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 5663fa6a1bb148c3afc9340477b05a4b (DEFAULT_TABLE table=default.Table_6e898d8b2bda40a8b235d59bc101c5e5 [id=9720988d247f4a2fbd324a101ed21590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.653244 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b1a27f687b54bc592ef4efb72b7247f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.668465 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5663fa6a1bb148c3afc9340477b05a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.669195 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_65d61963e9a74c85b88df60f7ebb82d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:18.671347 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65d61963e9a74c85b88df60f7ebb82d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:18.678360 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d6f23c83ff0a4358a7f096203284c706" } new_table_name: "default.Table_9a37007965804c1ea56673789a63e22b" modify_external_catalogs: true
I20260808 13:35:18.692190 6585 raft_consensus.cc:2229] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:18.692871 6585 raft_consensus.cc:2258] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:18.695077 6585 ts_tablet_manager.cc:1905] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:18.699926 4839 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1947f03e107447bb8e23fe5deaf1ea60 consensus state has no leader
I20260808 13:35:18.654937 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8cf06462484c99a860762a82f7ad64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.660853 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "543debff6da54d26bf62260bf8721ff7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:18.716938 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:18.718649 5834 tablet_service.cc:1467] Processing CreateTablet for tablet daa59a18ae624bc8975bee2c460895a7 (DEFAULT_TABLE table=default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.720335 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa59a18ae624bc8975bee2c460895a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.728371 6033 tablet_bootstrap.cc:492] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:18.728808 6033 ts_tablet_manager.cc:1397] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.639s user 0.010s sys 0.012s
W20260808 13:35:18.716485 5642 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.643s user 0.014s sys 0.042s
I20260808 13:35:18.738292 6615 raft_consensus.cc:482] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:18.606246 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b1a27f687b54bc592ef4efb72b7247f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.606359 6680 raft_consensus.cc:482] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:18.745391 6680 raft_consensus.cc:504] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.748607 6680 leader_election.cc:288] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:18.631470 5346 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:18.643635 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:18.689004 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 5663fa6a1bb148c3afc9340477b05a4b (DEFAULT_TABLE table=default.Table_6e898d8b2bda40a8b235d59bc101c5e5 [id=9720988d247f4a2fbd324a101ed21590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.690413 5355 tablet_service.cc:1467] Processing CreateTablet for tablet daa59a18ae624bc8975bee2c460895a7 (DEFAULT_TABLE table=default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.750918 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8cf06462484c99a860762a82f7ad64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.694200 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "543debff6da54d26bf62260bf8721ff7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:18.752182 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5663fa6a1bb148c3afc9340477b05a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.727495 5638 raft_consensus.cc:3037] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:18.754935 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa59a18ae624bc8975bee2c460895a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.755951 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 023fe27fc8ce4321a43a1f6037dd9cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.763223 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_00b62ebc438b40209bd662e8cd04fb29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:18.765313 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00b62ebc438b40209bd662e8cd04fb29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:18.767210 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_48ee76308d8846b49515e00b6bf24244"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:18.769196 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48ee76308d8846b49515e00b6bf24244 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:18.715862 6587 ts_tablet_manager.cc:1918] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:18.777546 6587 log.cc:1196] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/e2a95712de254c13b3b6dc4fe5cb15f7
I20260808 13:35:18.778851 6587 ts_tablet_manager.cc:1939] T e2a95712de254c13b3b6dc4fe5cb15f7 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:18.780129 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 128) took 1406 ms. Trace:
I20260808 13:35:18.781075 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 3a27517560d64a3b8da2cf55980e18a8 (DEFAULT_TABLE table=default.Table_0a3697cdad3740a1baa47b3075b5e1f9 [id=19a3a9ca647c4e069a20960c7f4cc57a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.782466 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a27517560d64a3b8da2cf55980e18a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.784492 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29786cb404bc2b01473c3b8390a50" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:18.782725 5600 tablet_service.cc:1467] Processing CreateTablet for tablet daa59a18ae624bc8975bee2c460895a7 (DEFAULT_TABLE table=default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.731618 5337 tablet_service.cc:1467] Processing CreateTablet for tablet 3a27517560d64a3b8da2cf55980e18a8 (DEFAULT_TABLE table=default.Table_0a3697cdad3740a1baa47b3075b5e1f9 [id=19a3a9ca647c4e069a20960c7f4cc57a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.735272 5338 tablet_service.cc:1467] Processing CreateTablet for tablet 362dbd6990e740b39690537af98aa2f8 (DEFAULT_TABLE table=default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:18.725072 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 626986us
I20260808 13:35:18.726039 6046 tablet_bootstrap.cc:492] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:18.796059 6046 ts_tablet_manager.cc:1397] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.734s user 0.018s sys 0.002s
I20260808 13:35:18.798238 6026 tablet_bootstrap.cc:492] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:18.798827 6026 ts_tablet_manager.cc:1397] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.561s user 0.015s sys 0.017s
W20260808 13:35:18.799008 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet adf9a03742a14cfe867c578c466838de consensus state has no leader
I20260808 13:35:18.800063 6328 leader_election.cc:288] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:18.804004 5771 leader_election.cc:341] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:18.802475 6033 raft_consensus.cc:348] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.805524 6033 raft_consensus.cc:374] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:18.805797 6033 raft_consensus.cc:729] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:18.538482 5879 raft_consensus.cc:2454] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:18.806758 6033 consensus_queue.cc:260] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.808748 6033 ts_tablet_manager.cc:1428] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.080s user 0.007s sys 0.000s
I20260808 13:35:18.809537 6033 tablet_bootstrap.cc:492] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:18.767750 6615 raft_consensus.cc:504] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.813493 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa59a18ae624bc8975bee2c460895a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.814267 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a27517560d64a3b8da2cf55980e18a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.816089 5606 tablet_service.cc:1467] Processing CreateTablet for tablet 5663fa6a1bb148c3afc9340477b05a4b (DEFAULT_TABLE table=default.Table_6e898d8b2bda40a8b235d59bc101c5e5 [id=9720988d247f4a2fbd324a101ed21590]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:18.817893 4823 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c -> Peer 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135): Couldn't send request to peer 87e8b888041b4135af55792050defcb5. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260808 13:35:18.823719 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 3a27517560d64a3b8da2cf55980e18a8 (DEFAULT_TABLE table=default.Table_0a3697cdad3740a1baa47b3075b5e1f9 [id=19a3a9ca647c4e069a20960c7f4cc57a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.824715 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a27517560d64a3b8da2cf55980e18a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.827355 4837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_0a3697cdad3740a1baa47b3075b5e1f9" } modify_external_catalogs: true
I20260808 13:35:18.827896 4837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_0a3697cdad3740a1baa47b3075b5e1f9" } modify_external_catalogs: true
I20260808 13:35:18.792099 6587 rpcz_store.cc:276] 0808 13:35:17.374020 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.374147 (+ 127us) service_pool.cc:227] Handling call
0808 13:35:17.708091 (+333944us) ts_tablet_manager.cc:1145] Deleting tablet e2a95712de254c13b3b6dc4fe5cb15f7
0808 13:35:17.708130 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:18.243954 (+535824us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:18.715598 (+471644us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:18.780021 (+ 64423us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":120798,"tablet-delete.queue_time_us":266836}
I20260808 13:35:18.832377 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet e2a95712de254c13b3b6dc4fe5cb15f7 (table default.Table_0a3697cdad3740a1baa47b3075b5e1f9 [id=19a3a9ca647c4e069a20960c7f4cc57a]) successfully deleted
I20260808 13:35:18.832842 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_bf8907064282450eb869d5bb8ec9f1c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:18.833374 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_810abec89b21407191098eb04ea650e2" } modify_external_catalogs: true
I20260808 13:35:18.833869 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_810abec89b21407191098eb04ea650e2" } modify_external_catalogs: true
W20260808 13:35:18.834378 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf8907064282450eb869d5bb8ec9f1c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:18.836603 6615 leader_election.cc:288] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:18.838291 5644 raft_consensus.cc:2454] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:18.839821 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5663fa6a1bb148c3afc9340477b05a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.839686 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29786cb404bc2b01473c3b8390a50" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:18.840556 5386 raft_consensus.cc:2454] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:18.840798 5285 leader_election.cc:302] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:18.841938 5551 leader_election.cc:302] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:18.842092 6680 raft_consensus.cc:2781] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:18.842512 6680 raft_consensus.cc:482] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:18.843173 6680 raft_consensus.cc:3037] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:18.843390 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 129) took 1347 ms. Trace:
I20260808 13:35:18.846814 5608 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50600 (request call id 142) took 1199 ms. Trace:
I20260808 13:35:18.847440 5608 rpcz_store.cc:276] 0808 13:35:17.647744 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.647885 (+ 141us) service_pool.cc:227] Handling call
0808 13:35:17.649678 (+ 1793us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:17.649791 (+ 113us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:18.259344 (+609553us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:18.743821 (+484477us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:18.846681 (+102860us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:18.870738 6706 raft_consensus.cc:2781] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:18.871400 6706 raft_consensus.cc:482] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:18.871971 6706 raft_consensus.cc:3037] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:18.844080 6587 rpcz_store.cc:276] 0808 13:35:17.502207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.502559 (+ 352us) service_pool.cc:227] Handling call
0808 13:35:18.842752 (+1340193us) ts_tablet_manager.cc:1145] Deleting tablet e00f2e0b98af431d99894e8778257bc4
0808 13:35:18.842827 (+ 75us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:18.843311 (+ 484us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1339401}
I20260808 13:35:18.874239 5338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362dbd6990e740b39690537af98aa2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:18.874387 6026 raft_consensus.cc:348] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:18.877219 6026 raft_consensus.cc:374] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:18.879473 6026 raft_consensus.cc:729] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:18.907212 4838 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_65d61963e9a74c85b88df60f7ebb82d2" } new_table_name: "default.Table_d553a5c837764412ab01800bea0017c2" modify_external_catalogs: true
I20260808 13:35:18.912457 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_bfdbf59c58ea43f1838a2d2a6b8102f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:18.914088 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bfdbf59c58ea43f1838a2d2a6b8102f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:18.928447 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 362dbd6990e740b39690537af98aa2f8 (DEFAULT_TABLE table=default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:18.931471 5609 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50600 (request call id 138) took 1307 ms. Trace:
I20260808 13:35:18.932090 5609 rpcz_store.cc:276] 0808 13:35:17.623711 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.626119 (+ 2408us) service_pool.cc:227] Handling call
0808 13:35:17.629659 (+ 3540us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:17.629780 (+ 121us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:18.563939 (+934159us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:18.919131 (+355192us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:18.931379 (+ 12248us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:18.773221 5382 rpcz_store.cc:276] 0808 13:35:17.424203 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.435272 (+ 11069us) service_pool.cc:227] Handling call
0808 13:35:17.436795 (+ 1523us) raft_consensus.cc:1457] Updating replica for 1 ops
0808 13:35:17.924833 (+488038us) raft_consensus.cc:1520] Early marking committed up to index 0
0808 13:35:17.924870 (+ 37us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0808 13:35:17.929115 (+ 4245us) log.cc:842] Serialized 26 byte log entry
0808 13:35:18.097592 (+168477us) raft_consensus.cc:1631] Marking committed up to 1
0808 13:35:18.241684 (+144092us) log.cc:842] Serialized 14 byte log entry
0808 13:35:18.455926 (+214242us) raft_consensus.cc:1683] Filling consensus response to leader.
0808 13:35:18.588015 (+132089us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54001216d4. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c27a2bf51 00007f5c27a2a99e 00007f5c28a072e9 00007f5c271dd684 00007f5c271dd612 00007f5c271dd541 00007f5c271dd4bc 00007f5c271dc772 00007f5c257da94c 00007f5c257d9d86
0808 13:35:18.588056 (+ 41us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0808 13:35:18.588113 (+ 57us) raft_consensus.cc:1671] finished
0808 13:35:18.588242 (+ 129us) raft_consensus.cc:1677] UpdateReplicas() finished
0808 13:35:18.588417 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":325983,"prepare.queue_time_us":143721,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":200376,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":391,"spinlock_wait_cycles":1121395456,"thread_start_us":100708,"threads_started":1,"wal-append.queue_time_us":67511}
I20260808 13:35:18.945720 6026 consensus_queue.cc:260] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.013394 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 1 got 0
I20260808 13:35:18.930424 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362dbd6990e740b39690537af98aa2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:19.015053 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:19.015908 4841 catalog_manager.cc:5495] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:19.018440 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 2 got 0
I20260808 13:35:19.021037 6046 raft_consensus.cc:348] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.022312 4839 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_0af988c3a5a243daa24e072c08164aaa" } new_table_name: "default.Table_256bb136799848f4891c3923692e5feb" modify_external_catalogs: true
I20260808 13:35:19.027077 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:19.030757 4837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 3a27517560d64a3b8da2cf55980e18a8
I20260808 13:35:19.031564 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 52deb2e968c04b99a436e04ae78d9969 (DEFAULT_TABLE table=default.Table_b2521a9f129840958fb2b9097b7cf37b [id=2256103e25f04641bd14f96720840445]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:19.035019 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet e00f2e0b98af431d99894e8778257bc4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e00f2e0b98af431d99894e8778257bc4 already in progress: creating tablet
I20260808 13:35:19.036033 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 6d8018bf8ff843d8818fbba9473acf47
I20260808 13:35:18.994254 5638 raft_consensus.cc:2454] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:19.046765 6026 ts_tablet_manager.cc:1428] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.248s user 0.008s sys 0.001s
I20260808 13:35:19.047816 6026 tablet_bootstrap.cc:492] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:19.064312 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a27517560d64a3b8da2cf55980e18a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:19.065114 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 362dbd6990e740b39690537af98aa2f8 (DEFAULT_TABLE table=default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:19.066015 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a27517560d64a3b8da2cf55980e18a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:19.066145 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362dbd6990e740b39690537af98aa2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:19.064218 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d8018bf8ff843d8818fbba9473acf47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:19.071995 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52deb2e968c04b99a436e04ae78d9969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:19.078225 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6e6ce417cc45fd90f5858c30d37a47" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:19.082769 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d8018bf8ff843d8818fbba9473acf47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:19.083679 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29786cb404bc2b01473c3b8390a50" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:19.084450 5879 raft_consensus.cc:2454] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:19.087170 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 138) took 1328 ms. Trace:
I20260808 13:35:19.087750 6587 rpcz_store.cc:276] 0808 13:35:17.759256 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.771312 (+ 12056us) service_pool.cc:227] Handling call
0808 13:35:19.037737 (+1266425us) ts_tablet_manager.cc:1145] Deleting tablet edfa66acceea4b54911d78ee16609ef7
0808 13:35:19.086778 (+ 49041us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:19.087080 (+ 302us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1101999}
I20260808 13:35:19.089288 6587 tablet_replica.cc:330] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:19.093421 5772 leader_election.cc:302] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:19.096095 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a27517560d64a3b8da2cf55980e18a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:19.098178 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d8018bf8ff843d8818fbba9473acf47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:19.022028 6046 raft_consensus.cc:374] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:19.084487 6591 ts_tablet_manager.cc:1918] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:19.107163 6591 log.cc:1196] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/edfa66acceea4b54911d78ee16609ef7
I20260808 13:35:19.108378 6591 ts_tablet_manager.cc:1939] T edfa66acceea4b54911d78ee16609ef7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:19.107981 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 52deb2e968c04b99a436e04ae78d9969 (DEFAULT_TABLE table=default.Table_b2521a9f129840958fb2b9097b7cf37b [id=2256103e25f04641bd14f96720840445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:19.109887 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52deb2e968c04b99a436e04ae78d9969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:19.113320 5642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.170.3:58777 (request call id 91) took 1069 ms. Trace:
I20260808 13:35:19.113864 5642 rpcz_store.cc:276] 0808 13:35:18.043579 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.072332 (+ 28753us) service_pool.cc:227] Handling call
0808 13:35:18.072902 (+ 570us) raft_consensus.cc:1457] Updating replica for 1 ops
0808 13:35:18.073209 (+ 307us) raft_consensus.cc:1520] Early marking committed up to index 0
0808 13:35:18.073238 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0808 13:35:18.073566 (+ 328us) alter_schema_op.cc:126] START. Timestamp: 1786196117250883
0808 13:35:18.892722 (+819156us) log.cc:842] Serialized 173 byte log entry
0808 13:35:19.084322 (+191600us) raft_consensus.cc:1631] Marking committed up to 2
0808 13:35:19.086841 (+ 2519us) raft_consensus.cc:1683] Filling consensus response to leader.
0808 13:35:19.093762 (+ 6921us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0808 13:35:19.112964 (+ 19202us) raft_consensus.cc:1671] finished
0808 13:35:19.113001 (+ 37us) raft_consensus.cc:1677] UpdateReplicas() finished
0808 13:35:19.113213 (+ 212us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:18.474138 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:18.487313 (+ 13175us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:18.487374 (+ 61us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:18.492149 (+ 4775us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
Metrics: {"thread_start_us":171770,"threads_started":1,"wal-append.queue_time_us":206518,"child_traces":[["op",{"prepare.queue_time_us":361774,"prepare.run_cpu_time_us":1461,"prepare.run_wall_time_us":37266,"thread_start_us":662491,"threads_started":1}]]}
I20260808 13:35:19.118088 6706 raft_consensus.cc:504] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.127529 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_0616044c71e04d1a83b3ffd25b84f54e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:19.129587 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0616044c71e04d1a83b3ffd25b84f54e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:19.102869 6046 raft_consensus.cc:729] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:19.141222 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_fd822a88bb39430eb487b2bdc43e86fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:19.143175 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd822a88bb39430eb487b2bdc43e86fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:19.133631 6046 consensus_queue.cc:260] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.153688 6026 tablet_bootstrap.cc:654] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:19.158684 6706 leader_election.cc:288] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:19.159184 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 120) took 2636 ms. Trace:
I20260808 13:35:19.159693 6591 rpcz_store.cc:276] 0808 13:35:16.523237 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.523368 (+ 131us) service_pool.cc:227] Handling call
0808 13:35:17.390922 (+867554us) ts_tablet_manager.cc:1145] Deleting tablet edfa66acceea4b54911d78ee16609ef7
0808 13:35:17.391003 (+ 81us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:18.598559 (+1207556us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.058112 (+459553us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.159050 (+100938us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":866819}
I20260808 13:35:19.161985 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet edfa66acceea4b54911d78ee16609ef7 (table default.Table_b034596e0aa1406ab5a3a849c74a03cc [id=62a056a39a41407e99810bcb504a91f5]) successfully deleted
I20260808 13:35:19.163304 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 52deb2e968c04b99a436e04ae78d9969 (DEFAULT_TABLE table=default.Table_b2521a9f129840958fb2b9097b7cf37b [id=2256103e25f04641bd14f96720840445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:19.164373 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52deb2e968c04b99a436e04ae78d9969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:19.090600 6587 raft_consensus.cc:2229] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:19.178864 6046 ts_tablet_manager.cc:1428] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.382s user 0.010s sys 0.003s
I20260808 13:35:19.026289 6680 raft_consensus.cc:504] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:19.188153 6728 log.cc:924] Time spent T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c: Append to log took a long time: real 0.090s user 0.001s sys 0.000s
I20260808 13:35:19.184978 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344828b16cfc47d5b419f8b569055fa4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:19.196524 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29786cb404bc2b01473c3b8390a50" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:19.197264 5878 raft_consensus.cc:2454] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:19.199753 6604 raft_consensus.cc:2781] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:19.200332 6604 raft_consensus.cc:686] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:19.201627 6604 consensus_queue.cc:237] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:19.204953 4838 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be has no consensus state
I20260808 13:35:19.162460 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 124) took 2559 ms. Trace:
I20260808 13:35:19.206588 6591 rpcz_store.cc:276] 0808 13:35:16.603096 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.603240 (+ 144us) service_pool.cc:227] Handling call
0808 13:35:19.161862 (+2558622us) ts_tablet_manager.cc:1145] Deleting tablet 9260370cf8cb46a7b699581e4e8d46db
0808 13:35:19.161969 (+ 107us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:19.162369 (+ 400us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2557922}
I20260808 13:35:19.207904 6680 leader_election.cc:288] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:19.192227 5550 consensus_peers.cc:487] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:19.208328 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 9260370cf8cb46a7b699581e4e8d46db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9260370cf8cb46a7b699581e4e8d46db already in progress: creating tablet
I20260808 13:35:19.201350 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29786cb404bc2b01473c3b8390a50" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:19.219798 5638 raft_consensus.cc:2379] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:19.216823 6706 raft_consensus.cc:482] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:19.222040 6706 raft_consensus.cc:504] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.228642 6033 tablet_bootstrap.cc:654] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:19.233912 6591 tablet_replica.cc:330] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:19.249917 6591 raft_consensus.cc:2229] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:19.250507 6591 raft_consensus.cc:2258] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:19.251112 6587 raft_consensus.cc:2258] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:19.252861 6591 ts_tablet_manager.cc:1905] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:19.253278 6587 ts_tablet_manager.cc:1905] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:19.272382 5325 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:46324 (request call id 12) took 1794 ms. Trace:
I20260808 13:35:19.272802 5325 rpcz_store.cc:276] 0808 13:35:17.477812 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.481823 (+ 4011us) service_pool.cc:227] Handling call
0808 13:35:17.985734 (+503911us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b5c0004210c. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c27dc0361 00007f5c27f5a1f9 00007f5c27e4c27b 00007f5c28a116a0 00007f5c272b9e04 00007f5c272b9d92 00007f5c272b9cc1 00007f5c272b9c3c 00007f5c272b8ef2 00007f5c257da94c 00007f5c257d9d86
0808 13:35:18.457352 (+471618us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54001216d4. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c27a25268 00007f5c28a116cd 00007f5c272b9e04 00007f5c272b9d92 00007f5c272b9cc1 00007f5c272b9c3c 00007f5c272b8ef2 00007f5c257da94c 00007f5c257d9d86 00007f5c257d972f
0808 13:35:19.272300 (+814948us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1101265536}
I20260808 13:35:19.288206 6706 leader_election.cc:288] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:19.288901 4835 master_service.cc:946] ReplaceTablet: received request to replace tablet e7616d81aa924933a9b24d44004725fa from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:19.293896 6046 tablet_bootstrap.cc:492] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:19.295514 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 202182us
I20260808 13:35:19.298120 5283 leader_election.cc:302] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:19.299315 6648 raft_consensus.cc:2726] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260808 13:35:19.304920 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet edfa66acceea4b54911d78ee16609ef7 because the tablet was not found. No further retry: Not found: Tablet not found: edfa66acceea4b54911d78ee16609ef7
I20260808 13:35:19.304677 6742 tablet.cc:1722] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:19.305812 6744 tablet.cc:1722] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:19.309633 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29786cb404bc2b01473c3b8390a50" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:19.310495 5384 raft_consensus.cc:2379] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:19.314148 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fb3e099f7f4544b3c100070f0d14ef" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:19.316632 6026 tablet_bootstrap.cc:492] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:19.317093 6026 ts_tablet_manager.cc:1397] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.269s user 0.009s sys 0.014s
I20260808 13:35:19.319679 6026 raft_consensus.cc:348] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.322822 6033 tablet_bootstrap.cc:492] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:19.323290 6033 ts_tablet_manager.cc:1397] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.514s user 0.013s sys 0.002s
I20260808 13:35:19.323791 5324 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:46324 (request call id 11) took 1872 ms. Trace:
I20260808 13:35:19.324216 5324 rpcz_store.cc:276] 0808 13:35:17.457415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.460432 (+ 3017us) service_pool.cc:227] Handling call
0808 13:35:18.110438 (+650006us) spinlock_profiling.cc:243] Waited 8.55 ms on lock 0x7b5c0004210c. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c27dc0361 00007f5c27f59fa9 00007f5c27e4c22a 00007f5c28a116a0 00007f5c272b9e04 00007f5c272b9d92 00007f5c272b9cc1 00007f5c272b9c3c 00007f5c272b8ef2 00007f5c257da94c 00007f5c257d9d86
0808 13:35:18.459315 (+348877us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54001216d4. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c27a25268 00007f5c28a116cd 00007f5c272b9e04 00007f5c272b9d92 00007f5c272b9cc1 00007f5c272b9c3c 00007f5c272b8ef2 00007f5c257da94c 00007f5c257d9d86 00007f5c257d972f
0808 13:35:19.310533 (+851218us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":796117120}
I20260808 13:35:19.320849 6026 raft_consensus.cc:374] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:19.330931 6026 raft_consensus.cc:729] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:19.331780 6026 consensus_queue.cc:260] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.336476 4838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6f4c6712b74748f58390a05ec941a26a" } modify_external_catalogs: true
I20260808 13:35:19.337065 4838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6f4c6712b74748f58390a05ec941a26a" } modify_external_catalogs: true
I20260808 13:35:19.339439 6591 ts_tablet_manager.cc:1918] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:19.340421 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1fe1ec319d834d319cd9c35e801b1c91" } modify_external_catalogs: true
I20260808 13:35:19.341089 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1fe1ec319d834d319cd9c35e801b1c91" } modify_external_catalogs: true
I20260808 13:35:19.350800 6591 log.cc:1196] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/2fa77744bcfb4a5cace730fd9f9fd1ca
I20260808 13:35:19.351780 6591 ts_tablet_manager.cc:1939] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:19.351267 5283 leader_election.cc:302] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:19.359050 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29786cb404bc2b01473c3b8390a50" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:19.360960 4823 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be (table default.Table_d553a5c837764412ab01800bea0017c2 [id=81ee34de9d5c494fb2bd74557a174c14], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be has no consensus state
I20260808 13:35:19.365604 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 125) took 2762 ms. Trace:
I20260808 13:35:19.366425 6591 rpcz_store.cc:276] 0808 13:35:16.603465 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.603599 (+ 134us) service_pool.cc:227] Handling call
0808 13:35:19.233553 (+2629954us) ts_tablet_manager.cc:1145] Deleting tablet 2fa77744bcfb4a5cace730fd9f9fd1ca
0808 13:35:19.233680 (+ 127us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:19.298949 (+ 65269us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.339147 (+ 40198us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.365530 (+ 26383us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":18300,"tablet-delete.queue_time_us":2629057}
I20260808 13:35:19.376050 6653 raft_consensus.cc:2726] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:19.376648 6591 tablet_replica.cc:330] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:19.377862 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 2fa77744bcfb4a5cace730fd9f9fd1ca (table default.Table_1f0431ee90fa4e2193747a84078f912c [id=c16baabb989244459aec3f7506aaa959]) successfully deleted
I20260808 13:35:19.375051 6033 raft_consensus.cc:348] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.378671 6033 raft_consensus.cc:374] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:19.378938 6033 raft_consensus.cc:729] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:19.383472 6591 raft_consensus.cc:2229] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:19.391806 6591 raft_consensus.cc:2258] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:19.362723 5878 raft_consensus.cc:3037] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:19.371465 6026 ts_tablet_manager.cc:1428] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.054s user 0.005s sys 0.001s
I20260808 13:35:19.399670 6026 tablet_bootstrap.cc:492] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:19.413121 6046 tablet_bootstrap.cc:654] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:19.414253 6591 ts_tablet_manager.cc:1905] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:19.417506 5386 raft_consensus.cc:3037] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:19.418428 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29786cb404bc2b01473c3b8390a50" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:19.379511 6033 consensus_queue.cc:260] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.424067 6033 ts_tablet_manager.cc:1428] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.100s user 0.006s sys 0.000s
I20260808 13:35:19.425889 5283 leader_election.cc:302] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:19.427251 6653 raft_consensus.cc:2726] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 13:35:19.428104 4833 master_service.cc:946] ReplaceTablet: received request to replace tablet d3682c9c656c4556bc13ae3aeddd7c5e from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:19.431644 6706 raft_consensus.cc:482] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:19.434944 6706 raft_consensus.cc:504] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.438658 6706 leader_election.cc:288] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:19.452419 6033 tablet_bootstrap.cc:492] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:19.455258 6026 tablet_bootstrap.cc:654] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:19.460031 5551 leader_election.cc:341] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:19.464079 4839 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d8e4af746774747a70d425068e8dd63 has no consensus state
I20260808 13:35:19.468802 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fb3e099f7f4544b3c100070f0d14ef" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:19.475236 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd2d0c9ab2c4704a74d06559bb913d6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:19.477512 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_3585f6d6de0e451e9732239606d0e39c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:19.480760 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3585f6d6de0e451e9732239606d0e39c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:19.481707 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd2d0c9ab2c4704a74d06559bb913d6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:19.498723 4842 catalog_manager.cc:5495] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:19.500126 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 1 got 0
I20260808 13:35:19.500823 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:19.501435 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 2 got 0
W20260808 13:35:19.512246 5551 leader_election.cc:341] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:19.487140 6706 consensus_queue.cc:1035] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
W20260808 13:35:19.519486 5550 leader_election.cc:341] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:19.520584 6759 tablet.cc:1722] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:19.523459 6587 ts_tablet_manager.cc:1918] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:19.520099 5550 leader_election.cc:302] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:19.527515 5550 leader_election.cc:341] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:19.528003 5550 leader_election.cc:302] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:19.528936 6729 raft_consensus.cc:2726] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:19.529517 6729 consensus_queue.cc:1035] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:19.540256 6585 ts_tablet_manager.cc:1918] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:19.540695 6585 log.cc:1196] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/cfc9a6ec9a86431fb46e50fcaa658d33
I20260808 13:35:19.541515 6585 ts_tablet_manager.cc:1939] T cfc9a6ec9a86431fb46e50fcaa658d33 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:19.537861 6706 raft_consensus.cc:2726] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:19.551029 4836 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_9b3bbebdc1274b238c30a45d36cf86f7" } new_table_name: "default.Table_a5c2164ca2344a5fa5cf5b85ddf5dfe5" modify_external_catalogs: true
I20260808 13:35:19.547348 6587 log.cc:1196] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/2fa77744bcfb4a5cace730fd9f9fd1ca
I20260808 13:35:19.560724 6587 ts_tablet_manager.cc:1939] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:19.565465 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 139) took 1805 ms. Trace:
I20260808 13:35:19.566161 6033 tablet_bootstrap.cc:654] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:19.577759 6762 tablet.cc:1722] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:19.579624 5878 raft_consensus.cc:2454] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:19.585878 6046 tablet_bootstrap.cc:492] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
W20260808 13:35:19.588176 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 60682.8us
I20260808 13:35:19.590339 5550 leader_election.cc:302] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:19.590902 6680 consensus_queue.cc:1035] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:19.593585 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet e992a0294c404cfb9541b57782a05fc9
I20260808 13:35:19.596192 6615 raft_consensus.cc:2781] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:19.597604 5833 tablet_service.cc:1514] Processing DeleteTablet for tablet e992a0294c404cfb9541b57782a05fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:19.597787 6615 raft_consensus.cc:686] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:19.598668 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet e992a0294c404cfb9541b57782a05fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:19.599303 6615 consensus_queue.cc:237] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.600726 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet e992a0294c404cfb9541b57782a05fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:19.609851 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 124) took 2985 ms. Trace:
I20260808 13:35:19.611052 6026 tablet_bootstrap.cc:492] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:19.611482 6026 ts_tablet_manager.cc:1397] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.212s user 0.016s sys 0.003s
I20260808 13:35:19.615161 6680 consensus_queue.cc:1035] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:19.615974 6591 ts_tablet_manager.cc:1918] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:19.615831 6026 raft_consensus.cc:348] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.616573 6026 raft_consensus.cc:374] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:19.616950 6026 raft_consensus.cc:729] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:19.646716 6046 ts_tablet_manager.cc:1397] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.353s user 0.015s sys 0.000s
I20260808 13:35:19.617827 6026 consensus_queue.cc:260] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.616524 6591 log.cc:1196] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/cfc9a6ec9a86431fb46e50fcaa658d33
I20260808 13:35:19.661741 6026 ts_tablet_manager.cc:1428] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.050s user 0.005s sys 0.001s
I20260808 13:35:19.662904 6591 ts_tablet_manager.cc:1939] T cfc9a6ec9a86431fb46e50fcaa658d33 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:19.663198 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_291e70645d0a4f24b232391de2e62d9b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:19.665028 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_291e70645d0a4f24b232391de2e62d9b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:19.565815 6587 rpcz_store.cc:276] 0808 13:35:17.759559 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.784613 (+ 25054us) service_pool.cc:227] Handling call
0808 13:35:19.089099 (+1304486us) ts_tablet_manager.cc:1145] Deleting tablet 2fa77744bcfb4a5cace730fd9f9fd1ca
0808 13:35:19.089150 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:19.459073 (+369923us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.496787 (+ 37714us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.565401 (+ 68614us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1303072}
I20260808 13:35:19.678297 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 126) took 3074 ms. Trace:
I20260808 13:35:19.679273 6587 tablet_replica.cc:330] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:19.681475 6587 raft_consensus.cc:2229] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:19.681782 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 2fa77744bcfb4a5cace730fd9f9fd1ca (table default.Table_1f0431ee90fa4e2193747a84078f912c [id=c16baabb989244459aec3f7506aaa959]) successfully deleted
I20260808 13:35:19.682207 6587 raft_consensus.cc:2258] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:19.683086 6768 log.cc:924] Time spent T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c: Append to log took a long time: real 0.064s user 0.001s sys 0.000s
I20260808 13:35:19.610318 6585 rpcz_store.cc:276] 0808 13:35:16.624971 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.627901 (+ 2930us) service_pool.cc:227] Handling call
0808 13:35:18.628821 (+2000920us) ts_tablet_manager.cc:1145] Deleting tablet cfc9a6ec9a86431fb46e50fcaa658d33
0808 13:35:18.628867 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:19.214224 (+585357us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.301726 (+ 87502us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.609740 (+308014us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":61961,"tablet-delete.queue_time_us":1996225}
I20260808 13:35:19.687948 6585 tablet_replica.cc:330] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:19.688225 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet cfc9a6ec9a86431fb46e50fcaa658d33 (table default.Table_a5dcf9a6d69a4c1c94e6c32081b818b7 [id=73f986b821c348e5b9e6cd5a1896eb83]) successfully deleted
I20260808 13:35:19.688273 6742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:46328 (request call id 126) took 2591 ms. Trace:
I20260808 13:35:19.678853 6591 rpcz_store.cc:276] 0808 13:35:16.603962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.604083 (+ 121us) service_pool.cc:227] Handling call
0808 13:35:19.376469 (+2772386us) ts_tablet_manager.cc:1145] Deleting tablet cfc9a6ec9a86431fb46e50fcaa658d33
0808 13:35:19.376527 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:19.537330 (+160803us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.615644 (+ 78314us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.678214 (+ 62570us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2771760}
I20260808 13:35:19.695566 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet cfc9a6ec9a86431fb46e50fcaa658d33 (table default.Table_a5dcf9a6d69a4c1c94e6c32081b818b7 [id=73f986b821c348e5b9e6cd5a1896eb83]) successfully deleted
I20260808 13:35:19.697036 6026 tablet_bootstrap.cc:492] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:19.699967 6591 tablet_replica.cc:330] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:19.700990 6046 raft_consensus.cc:348] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.701628 6046 raft_consensus.cc:374] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:19.701910 6046 raft_consensus.cc:729] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:19.702534 6046 consensus_queue.cc:260] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.709318 6587 ts_tablet_manager.cc:1905] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:19.717329 6046 ts_tablet_manager.cc:1428] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.070s user 0.006s sys 0.000s
I20260808 13:35:19.722198 6591 raft_consensus.cc:2229] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:19.726704 6591 raft_consensus.cc:2258] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:19.726927 6759 tablet.cc:1717] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb: Skipping requested alter to schema version 2, tablet already version 2
I20260808 13:35:19.690474 6742 rpcz_store.cc:276] 0808 13:35:17.097189 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.097361 (+ 172us) service_pool.cc:227] Handling call
0808 13:35:19.688162 (+2590801us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:17.211818 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:17.212589 (+ 771us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:17.212649 (+ 60us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:17.212766 (+ 117us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:17.212983 (+ 217us) alter_schema_op.cc:126] START. Timestamp: 1786196117212826
0808 13:35:17.213000 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:17.464548 (+251548us) log.cc:842] Serialized 173 byte log entry
0808 13:35:17.509857 (+ 45309us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54000a2654. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c27a27198 00007f5c27e88ade 00007f5c27e882b8 00007f5c27e8cba1 00007f5c27e8cb59 00007f5c27e8cae9 00007f5c27e8cab1 00007f5c27e8bdad 0000000000580784 00000000005805b9
0808 13:35:19.191678 (+1681821us) op_driver.cc:444] REPLICATION: finished
0808 13:35:19.304577 (+112899us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:19.307249 (+ 2672us) log.cc:842] Serialized 5 byte log entry
0808 13:35:19.638139 (+330890us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.642961 (+ 4822us) log.cc:842] Serialized 14 byte log entry
0808 13:35:19.666572 (+ 23611us) alter_schema_op.cc:71] Released schema lock
0808 13:35:19.673152 (+ 6580us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":112619,"mutex_wait_us":42222,"prepare.queue_time_us":106302,"prepare.run_cpu_time_us":7431,"prepare.run_wall_time_us":298145,"replication_time_us":1978553,"spinlock_wait_cycles":551939840,"thread_start_us":205451,"threads_started":2,"wal-append.queue_time_us":29117}]]}
I20260808 13:35:19.733485 6046 tablet_bootstrap.cc:492] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:19.739883 6026 tablet_bootstrap.cc:654] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:19.755172 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 9423b380a62c467895e7fdff14756586
I20260808 13:35:19.756963 6591 ts_tablet_manager.cc:1905] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:19.759538 6033 tablet_bootstrap.cc:492] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:19.759965 6033 ts_tablet_manager.cc:1397] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.308s user 0.009s sys 0.005s
I20260808 13:35:19.763667 6585 raft_consensus.cc:2229] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:19.764213 6585 raft_consensus.cc:2258] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:19.766312 6585 ts_tablet_manager.cc:1905] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:19.777899 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 9423b380a62c467895e7fdff14756586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:19.791146 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet d3682c9c656c4556bc13ae3aeddd7c5e
I20260808 13:35:19.792639 4833 catalog_manager.cc:6565] ReplaceTablet: tablet d3682c9c656c4556bc13ae3aeddd7c5e deleted and replaced by tablet ba39706cff65404485959865fff1772a
I20260808 13:35:19.796720 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e with delete_type TABLET_DATA_DELETED (replaced by tablet ba39706cff65404485959865fff1772a) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:19.798888 5845 tablet_service.cc:1514] Processing DeleteTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e with delete_type TABLET_DATA_DELETED (replaced by tablet ba39706cff65404485959865fff1772a) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:19.806962 6762 tablet.cc:1717] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Skipping requested alter to schema version 2, tablet already version 2
I20260808 13:35:19.807538 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e with delete_type TABLET_DATA_DELETED (replaced by tablet ba39706cff65404485959865fff1772a) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:19.809952 6591 ts_tablet_manager.cc:1918] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:19.811813 6615 raft_consensus.cc:482] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:19.813761 6615 raft_consensus.cc:504] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.809768 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 9423b380a62c467895e7fdff14756586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:19.821518 6615 leader_election.cc:288] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:19.823092 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ac4c366363468aa62b73e0f1e04532" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:19.812077 6591 log.cc:1196] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/5b6e6ce417cc45fd90f5858c30d37a47
W20260808 13:35:19.824487 5550 leader_election.cc:341] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:19.826121 6615 raft_consensus.cc:482] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:19.762176 6033 raft_consensus.cc:348] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.826709 6033 raft_consensus.cc:374] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:19.826982 6033 raft_consensus.cc:729] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:19.826828 6615 raft_consensus.cc:504] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.827636 6033 consensus_queue.cc:260] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.832223 6615 leader_election.cc:288] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:19.832741 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2173e97590be43f9a24f55689d796269" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:19.824728 6591 ts_tablet_manager.cc:1939] T 5b6e6ce417cc45fd90f5858c30d37a47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:19.833758 6779 tablet.cc:1717] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c: Skipping requested alter to schema version 2, tablet already version 2
I20260808 13:35:19.833772 5878 raft_consensus.cc:2454] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:19.837421 6779 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:46328 (request call id 147) took 1612 ms. Trace:
I20260808 13:35:19.841425 5550 leader_election.cc:302] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:19.841859 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ac4c366363468aa62b73e0f1e04532" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:19.843060 6615 raft_consensus.cc:2781] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:19.843680 6615 raft_consensus.cc:482] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:19.843792 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 9423b380a62c467895e7fdff14756586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:19.844381 6615 raft_consensus.cc:3037] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:19.849182 5551 leader_election.cc:341] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:19.849668 5551 leader_election.cc:302] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:19.851262 6765 raft_consensus.cc:2726] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:19.865957 4823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260808 13:35:19.865818 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2173e97590be43f9a24f55689d796269" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:19.868366 6046 tablet_bootstrap.cc:654] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:19.837924 6779 rpcz_store.cc:276] 0808 13:35:18.224837 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.254749 (+ 29912us) service_pool.cc:227] Handling call
0808 13:35:19.837346 (+1582597us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:18.261797 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:18.262516 (+ 719us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:19.666266 (+1403750us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:19.669565 (+ 3299us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:19.669750 (+ 185us) alter_schema_op.cc:126] START. Timestamp: 1786196119669610
0808 13:35:19.669782 (+ 32us) op_driver.cc:331] REPLICATION: starting
0808 13:35:19.670444 (+ 662us) log.cc:842] Serialized 173 byte log entry
0808 13:35:19.720657 (+ 50213us) op_driver.cc:444] REPLICATION: finished
0808 13:35:19.743503 (+ 22846us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:19.837014 (+ 93511us) log.cc:842] Serialized 94 byte log entry
0808 13:35:19.837138 (+ 124us) alter_schema_op.cc:71] Released schema lock
0808 13:35:19.837194 (+ 56us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":22503,"prepare.queue_time_us":375,"prepare.run_cpu_time_us":53603,"prepare.run_wall_time_us":1415979,"raft.queue_time_us":277,"raft.run_cpu_time_us":1971,"raft.run_wall_time_us":2481,"replication_time_us":50786,"thread_start_us":13804,"threads_started":1}]]}
I20260808 13:35:19.870054 6033 ts_tablet_manager.cc:1428] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.110s user 0.005s sys 0.003s
W20260808 13:35:19.876868 5551 leader_election.cc:341] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:19.882675 6615 raft_consensus.cc:504] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:19.889323 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a730d953de56401597f5e28a6edc707a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:19.891342 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a730d953de56401597f5e28a6edc707a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:19.892588 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 127) took 3288 ms. Trace:
I20260808 13:35:19.893126 6591 rpcz_store.cc:276] 0808 13:35:16.604477 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.604700 (+ 223us) service_pool.cc:227] Handling call
0808 13:35:19.699644 (+3094944us) ts_tablet_manager.cc:1145] Deleting tablet 5b6e6ce417cc45fd90f5858c30d37a47
0808 13:35:19.699755 (+ 111us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:19.778654 (+ 78899us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.809618 (+ 30964us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.892479 (+ 82861us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3094064}
I20260808 13:35:19.894666 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 5b6e6ce417cc45fd90f5858c30d37a47 (table default.Table_ade5bcdd6da94c6284819d3fe37f039f [id=3d8a7488d67c47e291b817ee36e99ac0]) successfully deleted
I20260808 13:35:19.892602 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_e38e30a466f94219836c80d87f91c8a9" } new_table_name: "default.Table_880132219c91464db6a219d0e98711d5" modify_external_catalogs: true
I20260808 13:35:19.895177 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2173e97590be43f9a24f55689d796269" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:19.895968 5878 raft_consensus.cc:3037] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:19.893671 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6ba381f21fc24362bc6d18499af1d329" } modify_external_catalogs: true
I20260808 13:35:19.898846 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6ba381f21fc24362bc6d18499af1d329" } modify_external_catalogs: true
I20260808 13:35:19.899341 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 133) took 2842 ms. Trace:
I20260808 13:35:19.899901 6591 rpcz_store.cc:276] 0808 13:35:17.057390 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.057609 (+ 219us) service_pool.cc:227] Handling call
0808 13:35:19.898551 (+2840942us) ts_tablet_manager.cc:1145] Deleting tablet fc9b2e69fe27431dac9eb5ecb9e3e23e
0808 13:35:19.898636 (+ 85us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:19.899240 (+ 604us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2839691}
W20260808 13:35:19.901901 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc9b2e69fe27431dac9eb5ecb9e3e23e already in progress: creating tablet
I20260808 13:35:19.902715 6615 leader_election.cc:288] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:19.905757 5878 raft_consensus.cc:2454] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:19.905826 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_9fa5c49027684fe6a85a46895e121932"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:19.907447 5550 leader_election.cc:302] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:35:19.907735 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fa5c49027684fe6a85a46895e121932 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:19.908721 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 132) took 2885 ms. Trace:
I20260808 13:35:19.908465 6615 raft_consensus.cc:2781] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:19.911144 6033 tablet_bootstrap.cc:492] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:19.913139 6615 raft_consensus.cc:686] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:19.909204 6591 rpcz_store.cc:276] 0808 13:35:17.023496 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.023676 (+ 180us) service_pool.cc:227] Handling call
0808 13:35:19.906427 (+2882751us) ts_tablet_manager.cc:1145] Deleting tablet 127e8d0323954ef788018bcb91582951
0808 13:35:19.906506 (+ 79us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:19.908653 (+ 2147us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2823451}
I20260808 13:35:19.915051 6615 consensus_queue.cc:237] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:19.916141 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 127e8d0323954ef788018bcb91582951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 127e8d0323954ef788018bcb91582951 already in progress: creating tablet
W20260808 13:35:19.917059 4823 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:37135, user_credentials={real_user=slave}} blocked reactor thread for 51136us
I20260808 13:35:19.917779 6591 tablet_replica.cc:330] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:19.921057 6591 raft_consensus.cc:2229] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:19.923609 6591 pending_rounds.cc:62] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: Trying to abort 1 pending ops.
I20260808 13:35:19.923916 6591 pending_rounds.cc:68] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259305900990464 op_type: NO_OP noop_request { }
I20260808 13:35:19.935703 4835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet e7616d81aa924933a9b24d44004725fa
I20260808 13:35:19.937340 4835 catalog_manager.cc:6565] ReplaceTablet: tablet e7616d81aa924933a9b24d44004725fa deleted and replaced by tablet 1b9e8d48c55349058cf80bf7c2e149e7
I20260808 13:35:19.941246 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet e7616d81aa924933a9b24d44004725fa with delete_type TABLET_DATA_DELETED (replaced by tablet 1b9e8d48c55349058cf80bf7c2e149e7) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:19.924299 6591 raft_consensus.cc:2866] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:35:19.944944 6591 raft_consensus.cc:2258] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:19.945393 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet e7616d81aa924933a9b24d44004725fa with delete_type TABLET_DATA_DELETED (replaced by tablet 1b9e8d48c55349058cf80bf7c2e149e7) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:19.956097 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 1 got 0
I20260808 13:35:19.957831 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 2 got 0
W20260808 13:35:19.887624 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 158237us
I20260808 13:35:19.963254 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2173e97590be43f9a24f55689d796269" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:19.964989 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet e7616d81aa924933a9b24d44004725fa with delete_type TABLET_DATA_DELETED (replaced by tablet 1b9e8d48c55349058cf80bf7c2e149e7) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:19.966988 5551 leader_election.cc:341] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:19.975560 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 110354us
W20260808 13:35:19.986210 4842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 33) took 939 ms (client timeout 1000 ms). Trace:
I20260808 13:35:19.995924 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:19.998425 6587 ts_tablet_manager.cc:1918] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:19.998997 6587 log.cc:1196] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/cfc9a6ec9a86431fb46e50fcaa658d33
I20260808 13:35:19.997143 4835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_bfdbf59c58ea43f1838a2d2a6b8102f9" } modify_external_catalogs: true
I20260808 13:35:20.011822 4835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_bfdbf59c58ea43f1838a2d2a6b8102f9" } modify_external_catalogs: true
I20260808 13:35:20.018970 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 543debff6da54d26bf62260bf8721ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:20.020167 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:20.006260 6587 ts_tablet_manager.cc:1939] T cfc9a6ec9a86431fb46e50fcaa658d33 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
W20260808 13:35:19.986538 4842 rpcz_store.cc:269] 0808 13:35:19.046536 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.046771 (+ 235us) service_pool.cc:227] Handling call
0808 13:35:19.986147 (+939376us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:19.916470 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:19.916847 (+ 377us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:19.916871 (+ 24us) write_op.cc:435] Acquired schema lock
0808 13:35:19.916883 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:19.917143 (+ 260us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:19.917198 (+ 55us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:19.917220 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:19.917367 (+ 147us) tablet.cc:699] Row locks acquired
0808 13:35:19.917393 (+ 26us) write_op.cc:260] PREPARE: finished
0808 13:35:19.917521 (+ 128us) write_op.cc:270] Start()
0808 13:35:19.917609 (+ 88us) write_op.cc:276] Timestamp: P: 1786196119917474 usec, L: 0
0808 13:35:19.917624 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:19.918199 (+ 575us) log.cc:842] Serialized 527 byte log entry
0808 13:35:19.931704 (+ 13505us) op_driver.cc:444] REPLICATION: finished
0808 13:35:19.943033 (+ 11329us) write_op.cc:301] APPLY: starting
0808 13:35:19.943132 (+ 99us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:19.943300 (+ 168us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:19.943457 (+ 157us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:19.943815 (+ 358us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:19.944080 (+ 265us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:19.944154 (+ 74us) write_op.cc:312] APPLY: finished
0808 13:35:19.944358 (+ 204us) log.cc:842] Serialized 23 byte log entry
0808 13:35:19.946892 (+ 2534us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:19.947008 (+ 116us) write_op.cc:454] Released schema lock
0808 13:35:19.947063 (+ 55us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:19.948067 (+ 1004us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:19.958697 (+ 10630us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8518,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":11137,"apply.run_cpu_time_us":2426,"apply.run_wall_time_us":15972,"num_ops":1,"prepare.queue_time_us":407872,"prepare.run_cpu_time_us":2240,"prepare.run_wall_time_us":59775,"raft.queue_time_us":29193,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":494,"replication_time_us":14018}]]}
I20260808 13:35:20.023192 6734 consensus_queue.cc:1035] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:20.024096 6033 tablet_bootstrap.cc:654] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:20.027674 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 140) took 2241 ms. Trace:
I20260808 13:35:20.028251 6587 rpcz_store.cc:276] 0808 13:35:17.786502 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.786725 (+ 223us) service_pool.cc:227] Handling call
0808 13:35:19.679030 (+1892305us) ts_tablet_manager.cc:1145] Deleting tablet cfc9a6ec9a86431fb46e50fcaa658d33
0808 13:35:19.679100 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:19.872169 (+193069us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:19.998061 (+125892us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.027618 (+ 29557us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1891378}
I20260808 13:35:20.030326 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet cfc9a6ec9a86431fb46e50fcaa658d33 (table default.Table_a5dcf9a6d69a4c1c94e6c32081b818b7 [id=73f986b821c348e5b9e6cd5a1896eb83]) successfully deleted
I20260808 13:35:20.032630 6046 tablet_bootstrap.cc:492] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:20.033051 6046 ts_tablet_manager.cc:1397] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.315s user 0.012s sys 0.012s
I20260808 13:35:20.038597 6046 raft_consensus.cc:348] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.039374 6046 raft_consensus.cc:374] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:20.039601 6046 raft_consensus.cc:729] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:20.040235 6046 consensus_queue.cc:260] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.045050 6046 ts_tablet_manager.cc:1428] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260808 13:35:20.046203 6046 tablet_bootstrap.cc:492] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:20.049880 6591 ts_tablet_manager.cc:1905] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.062274 6046 tablet_bootstrap.cc:654] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:20.063812 4832 catalog_manager.cc:5495] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:20.090991 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:20.096472 6587 tablet_replica.cc:330] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:20.098008 6587 raft_consensus.cc:2229] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:20.101835 6587 raft_consensus.cc:2258] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:20.103184 6046 tablet_bootstrap.cc:492] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:20.103585 6046 ts_tablet_manager.cc:1397] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.058s user 0.016s sys 0.009s
I20260808 13:35:20.106138 6046 raft_consensus.cc:348] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.106808 6046 raft_consensus.cc:374] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:20.106992 6026 tablet_bootstrap.cc:492] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:20.107054 6046 raft_consensus.cc:729] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:20.107429 6026 ts_tablet_manager.cc:1397] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.445s user 0.038s sys 0.029s
I20260808 13:35:20.107633 6046 consensus_queue.cc:260] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.109247 4841 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_fd5435dd94cd493ab4780d4dea9de9a8 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7] alter complete (version 2)
I20260808 13:35:20.110433 6587 ts_tablet_manager.cc:1905] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.117337 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:20.118188 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a95712de254c13b3b6dc4fe5cb15f7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a27517560d64a3b8da2cf55980e18a8) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:20.119081 5353 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:20.120739 4836 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4b7284afe0294644a368cf51451c7ab6 consensus state has no leader
I20260808 13:35:20.122509 6046 ts_tablet_manager.cc:1428] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.019s user 0.005s sys 0.001s
I20260808 13:35:20.125574 6046 tablet_bootstrap.cc:492] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:20.132232 4841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 41) took 1289 ms (client timeout 1000 ms). Trace:
I20260808 13:35:20.135488 6793 raft_consensus.cc:482] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:20.136345 6793 raft_consensus.cc:504] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:20.132632 4841 rpcz_store.cc:269] 0808 13:35:18.842873 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.844296 (+ 1423us) service_pool.cc:227] Handling call
0808 13:35:20.132162 (+1287866us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:19.146722 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:19.147127 (+ 405us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:19.147158 (+ 31us) write_op.cc:435] Acquired schema lock
0808 13:35:19.147174 (+ 16us) tablet.cc:660] Decoding operations
0808 13:35:19.147458 (+ 284us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:19.147496 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:19.147527 (+ 31us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:19.147675 (+ 148us) tablet.cc:699] Row locks acquired
0808 13:35:19.147694 (+ 19us) write_op.cc:260] PREPARE: finished
0808 13:35:19.147821 (+ 127us) write_op.cc:270] Start()
0808 13:35:19.147914 (+ 93us) write_op.cc:276] Timestamp: P: 1786196119147743 usec, L: 0
0808 13:35:19.147930 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:19.148550 (+ 620us) log.cc:842] Serialized 527 byte log entry
0808 13:35:19.470996 (+322446us) op_driver.cc:444] REPLICATION: finished
0808 13:35:19.486929 (+ 15933us) write_op.cc:301] APPLY: starting
0808 13:35:19.487667 (+ 738us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:19.487927 (+ 260us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:19.488156 (+ 229us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:19.488479 (+ 323us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:19.488655 (+ 176us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:19.488707 (+ 52us) write_op.cc:312] APPLY: finished
0808 13:35:19.488981 (+ 274us) log.cc:842] Serialized 23 byte log entry
0808 13:35:19.489757 (+ 776us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:19.489921 (+ 164us) write_op.cc:454] Released schema lock
0808 13:35:19.489986 (+ 65us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:19.519001 (+ 29015us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:19.519052 (+ 51us) write_op.cc:454] Released schema lock
Related trace 'op':
0808 13:35:19.976456 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:19.984136 (+ 7680us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:19.984164 (+ 28us) write_op.cc:435] Acquired schema lock
0808 13:35:19.984178 (+ 14us) tablet.cc:660] Decoding operations
0808 13:35:19.988652 (+ 4474us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:19.988694 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:19.988716 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:19.988873 (+ 157us) tablet.cc:699] Row locks acquired
0808 13:35:19.988886 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:19.989005 (+ 119us) write_op.cc:270] Start()
0808 13:35:19.989115 (+ 110us) write_op.cc:276] Timestamp: P: 1786196119988948 usec, L: 0
0808 13:35:19.989131 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:19.989702 (+ 571us) log.cc:842] Serialized 356 byte log entry
0808 13:35:20.104106 (+114404us) op_driver.cc:444] REPLICATION: finished
0808 13:35:20.106343 (+ 2237us) write_op.cc:301] APPLY: starting
0808 13:35:20.106414 (+ 71us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:20.106503 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:20.106600 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:20.106801 (+ 201us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:20.106937 (+ 136us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:20.106981 (+ 44us) write_op.cc:312] APPLY: finished
0808 13:35:20.107166 (+ 185us) log.cc:842] Serialized 23 byte log entry
0808 13:35:20.107318 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:20.107400 (+ 82us) write_op.cc:454] Released schema lock
0808 13:35:20.107429 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:20.107603 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:20.107629 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":15626,"apply.run_cpu_time_us":2398,"apply.run_wall_time_us":32518,"num_ops":1,"prepare.queue_time_us":102331,"prepare.run_cpu_time_us":2230,"prepare.run_wall_time_us":62843,"raft.queue_time_us":192,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":15,"replication_time_us":322934}],["op",{"apply.queue_time_us":2068,"apply.run_cpu_time_us":1480,"apply.run_wall_time_us":1487,"num_ops":1,"prepare.queue_time_us":465822,"prepare.run_cpu_time_us":2361,"prepare.run_wall_time_us":33504,"raft.queue_time_us":164,"raft.run_cpu_time_us":789,"raft.run_wall_time_us":3602,"replication_time_us":114899}]]}
I20260808 13:35:20.145732 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "543debff6da54d26bf62260bf8721ff7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:20.155193 6026 raft_consensus.cc:348] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.156010 6026 raft_consensus.cc:374] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:20.156297 6026 raft_consensus.cc:729] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:20.157318 6793 leader_election.cc:288] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:20.164287 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "543debff6da54d26bf62260bf8721ff7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:20.168617 5283 leader_election.cc:302] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:20.174980 5382 raft_consensus.cc:3037] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:20.178857 6046 tablet_bootstrap.cc:654] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:20.171001 6026 consensus_queue.cc:260] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.190910 6026 ts_tablet_manager.cc:1428] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.083s user 0.007s sys 0.005s
I20260808 13:35:20.192257 6026 tablet_bootstrap.cc:492] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:20.193482 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_04a268476ef54bb68aadad0d39d6e838"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:20.195861 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04a268476ef54bb68aadad0d39d6e838 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:20.210892 6591 ts_tablet_manager.cc:1918] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:20.211334 6591 log.cc:1196] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/a0b29786cb404bc2b01473c3b8390a50
I20260808 13:35:20.212219 6591 ts_tablet_manager.cc:1939] T a0b29786cb404bc2b01473c3b8390a50 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:20.217628 6033 tablet_bootstrap.cc:492] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:20.218081 6033 ts_tablet_manager.cc:1397] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.309s user 0.008s sys 0.013s
I20260808 13:35:20.227200 6585 ts_tablet_manager.cc:1918] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:20.228034 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 134) took 3070 ms. Trace:
I20260808 13:35:20.227787 6585 log.cc:1196] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/5b6e6ce417cc45fd90f5858c30d37a47
I20260808 13:35:20.234349 6585 ts_tablet_manager.cc:1939] T 5b6e6ce417cc45fd90f5858c30d37a47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:20.238114 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 127) took 3479 ms. Trace:
I20260808 13:35:20.239475 6026 tablet_bootstrap.cc:654] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:20.252856 6793 raft_consensus.cc:2726] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:20.252645 6585 rpcz_store.cc:276] 0808 13:35:16.786634 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.790566 (+ 3932us) service_pool.cc:227] Handling call
0808 13:35:19.687360 (+2896794us) ts_tablet_manager.cc:1145] Deleting tablet 5b6e6ce417cc45fd90f5858c30d37a47
0808 13:35:19.687677 (+ 317us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.020589 (+332912us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.226787 (+206198us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.238030 (+ 11243us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2888033}
I20260808 13:35:20.254822 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 5b6e6ce417cc45fd90f5858c30d37a47 (table default.Table_ade5bcdd6da94c6284819d3fe37f039f [id=3d8a7488d67c47e291b817ee36e99ac0]) successfully deleted
I20260808 13:35:20.256711 6587 ts_tablet_manager.cc:1918] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:20.257078 6587 log.cc:1196] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/78d492406c8a4c4186131e02a1559cb4
I20260808 13:35:20.257725 6587 ts_tablet_manager.cc:1939] T 78d492406c8a4c4186131e02a1559cb4 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:20.228438 6591 rpcz_store.cc:276] 0808 13:35:17.157419 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.157589 (+ 170us) service_pool.cc:227] Handling call
0808 13:35:19.917519 (+2759930us) ts_tablet_manager.cc:1145] Deleting tablet a0b29786cb404bc2b01473c3b8390a50
0808 13:35:19.917621 (+ 102us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.151192 (+233571us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.170485 (+ 19293us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.227952 (+ 57467us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2747676}
I20260808 13:35:20.259649 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet a0b29786cb404bc2b01473c3b8390a50 (table default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]) successfully deleted
I20260808 13:35:20.260591 6591 tablet_replica.cc:330] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:20.261360 6591 raft_consensus.cc:2229] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:20.261901 6591 raft_consensus.cc:2258] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:20.263964 6591 ts_tablet_manager.cc:1905] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.264115 6734 consensus_queue.cc:1035] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:20.265524 6585 tablet_replica.cc:330] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:20.267099 6585 raft_consensus.cc:2229] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:20.267822 6585 raft_consensus.cc:2258] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:20.258448 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 141) took 2470 ms. Trace:
I20260808 13:35:20.273979 6033 raft_consensus.cc:348] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.281019 6587 rpcz_store.cc:276] 0808 13:35:17.788034 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.788169 (+ 135us) service_pool.cc:227] Handling call
0808 13:35:20.096015 (+2307846us) ts_tablet_manager.cc:1145] Deleting tablet 78d492406c8a4c4186131e02a1559cb4
0808 13:35:20.096081 (+ 66us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.175551 (+ 79470us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.256489 (+ 80938us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.258401 (+ 1912us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2306765}
I20260808 13:35:20.293953 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 78d492406c8a4c4186131e02a1559cb4 (table default.Table_6cb245b3d74d4347843035902b8d49d1 [id=c0806126263c41b68137cad5e21de125]) successfully deleted
W20260808 13:35:20.299530 4837 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8ba098c946794607b2a15bc64098d918 consensus state has no leader
I20260808 13:35:20.307081 6587 tablet_replica.cc:330] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:20.313410 6046 tablet_bootstrap.cc:492] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:20.314905 6046 ts_tablet_manager.cc:1397] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.189s user 0.013s sys 0.013s
I20260808 13:35:20.316028 6033 raft_consensus.cc:374] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:20.316361 6033 raft_consensus.cc:729] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:20.317044 6033 consensus_queue.cc:260] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.319643 6033 ts_tablet_manager.cc:1428] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.101s user 0.003s sys 0.005s
I20260808 13:35:20.331068 6587 raft_consensus.cc:2229] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:20.331939 6587 raft_consensus.cc:2258] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:20.331128 6585 ts_tablet_manager.cc:1905] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.334018 6046 raft_consensus.cc:348] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.334717 6046 raft_consensus.cc:374] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:20.335009 6046 raft_consensus.cc:729] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:20.335562 6046 consensus_queue.cc:260] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.337656 6046 ts_tablet_manager.cc:1428] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.022s user 0.000s sys 0.004s
I20260808 13:35:20.358050 6615 raft_consensus.cc:482] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:20.358722 6615 raft_consensus.cc:504] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.360464 6033 tablet_bootstrap.cc:492] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:20.362282 6615 leader_election.cc:288] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:20.363735 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4625b632d67244fc86e48d739dde707e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:20.364127 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_a5c2164ca2344a5fa5cf5b85ddf5dfe5 [id=26fb789fff434bb0825daf68d5378013], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4b7284afe0294644a368cf51451c7ab6 consensus state has no leader
I20260808 13:35:20.364974 5386 raft_consensus.cc:2454] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:20.365578 6046 tablet_bootstrap.cc:492] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:20.366495 5551 leader_election.cc:302] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:20.367807 6615 raft_consensus.cc:2781] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:20.368423 6615 raft_consensus.cc:482] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:20.371675 6615 raft_consensus.cc:3037] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:20.344524 6587 ts_tablet_manager.cc:1905] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.382893 6046 tablet_bootstrap.cc:654] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:20.383888 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 851a0b374a854efd87dd66e39b9df2d5
I20260808 13:35:20.390378 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 851a0b374a854efd87dd66e39b9df2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:20.386348 4833 master_service.cc:946] ReplaceTablet: received request to replace tablet 127e8d0323954ef788018bcb91582951 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:20.396276 6046 tablet_bootstrap.cc:492] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:20.396740 6046 ts_tablet_manager.cc:1397] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.007s
I20260808 13:35:20.400009 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_bb876d9598e24abf96366e835a8b2d59" } new_table_name: "default.Table_13ca344305b74220b866f3194259e283" modify_external_catalogs: true
I20260808 13:35:20.406306 6591 ts_tablet_manager.cc:1918] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:20.405833 6046 raft_consensus.cc:348] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.406663 6046 raft_consensus.cc:374] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:20.406949 6046 raft_consensus.cc:729] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:20.407250 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_03b5de95197845c1bc7ab06aec3d983d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:20.407784 6450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260808 13:35:20.407612 6046 consensus_queue.cc:260] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:20.408558 6450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:35:20.409238 6450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 13:35:20.409789 6046 ts_tablet_manager.cc:1428] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
W20260808 13:35:20.409931 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03b5de95197845c1bc7ab06aec3d983d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:20.410732 6046 tablet_bootstrap.cc:492] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:20.419003 6046 tablet_bootstrap.cc:654] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:20.411561 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 851a0b374a854efd87dd66e39b9df2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:20.424113 6591 log.cc:1196] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/e2a95712de254c13b3b6dc4fe5cb15f7
I20260808 13:35:20.425549 6591 ts_tablet_manager.cc:1939] T e2a95712de254c13b3b6dc4fe5cb15f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:20.429278 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 135) took 3219 ms. Trace:
I20260808 13:35:20.438885 6591 rpcz_store.cc:276] 0808 13:35:17.209730 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.210584 (+ 854us) service_pool.cc:227] Handling call
0808 13:35:20.260432 (+3049848us) ts_tablet_manager.cc:1145] Deleting tablet e2a95712de254c13b3b6dc4fe5cb15f7
0808 13:35:20.260475 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.379862 (+119387us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.406079 (+ 26217us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.429221 (+ 23142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3048770}
I20260808 13:35:20.435791 6831 tablet.cc:1722] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:20.442699 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 136) took 2919 ms. Trace:
I20260808 13:35:20.447360 6591 rpcz_store.cc:276] 0808 13:35:17.523360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.531221 (+ 7861us) service_pool.cc:227] Handling call
0808 13:35:20.442291 (+2911070us) ts_tablet_manager.cc:1145] Deleting tablet e00f2e0b98af431d99894e8778257bc4
0808 13:35:20.442345 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.442587 (+ 242us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2837741}
W20260808 13:35:20.455384 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet e00f2e0b98af431d99894e8778257bc4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e00f2e0b98af431d99894e8778257bc4 already in progress: creating tablet
I20260808 13:35:20.456652 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 851a0b374a854efd87dd66e39b9df2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:20.456755 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4625b632d67244fc86e48d739dde707e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:20.456589 6615 raft_consensus.cc:504] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.457044 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 145) took 2361 ms. Trace:
I20260808 13:35:20.457351 6591 rpcz_store.cc:276] 0808 13:35:18.095377 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.098745 (+ 3368us) service_pool.cc:227] Handling call
0808 13:35:20.456693 (+2357948us) ts_tablet_manager.cc:1145] Deleting tablet 345d13cbde6844079c2d1eb80732bb1f
0808 13:35:20.456737 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.456990 (+ 253us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2356988}
W20260808 13:35:20.458334 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 345d13cbde6844079c2d1eb80732bb1f because the tablet was not found. No further retry: Not found: Tablet not found: 345d13cbde6844079c2d1eb80732bb1f
I20260808 13:35:20.459355 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4625b632d67244fc86e48d739dde707e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:20.460145 5386 raft_consensus.cc:3037] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:20.465389 4839 master_service.cc:946] ReplaceTablet: received request to replace tablet 69642fdba1f145438b04eae9f8154101 from {username='slave'} at 127.0.0.1:60270
W20260808 13:35:20.466549 5550 leader_election.cc:341] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:20.473673 6591 tablet_replica.cc:330] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:20.475613 6591 raft_consensus.cc:2229] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:20.476513 6591 raft_consensus.cc:2258] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:20.484100 6591 ts_tablet_manager.cc:1905] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.489928 5386 raft_consensus.cc:2454] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:20.490940 5551 leader_election.cc:302] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:20.493547 6033 tablet_bootstrap.cc:654] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:20.495127 6765 raft_consensus.cc:2781] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:20.498430 6615 leader_election.cc:288] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:20.499789 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4625b632d67244fc86e48d739dde707e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:20.501585 5550 leader_election.cc:341] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:20.507876 6026 tablet_bootstrap.cc:492] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:20.508257 6026 ts_tablet_manager.cc:1397] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.316s user 0.021s sys 0.018s
I20260808 13:35:20.510560 6026 raft_consensus.cc:348] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.511226 6026 raft_consensus.cc:374] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:20.511461 6026 raft_consensus.cc:729] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:20.512205 6026 consensus_queue.cc:260] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.517848 6026 ts_tablet_manager.cc:1428] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260808 13:35:20.523106 6026 tablet_bootstrap.cc:492] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:20.539459 6026 tablet_bootstrap.cc:654] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:20.547776 6591 ts_tablet_manager.cc:1918] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:20.548238 6591 log.cc:1196] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:20.548830 6591 ts_tablet_manager.cc:1939] T 81fb3e099f7f4544b3c100070f0d14ef P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:20.550561 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 146) took 2454 ms. Trace:
I20260808 13:35:20.551924 6829 tablet.cc:1722] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:20.551054 6591 rpcz_store.cc:276] 0808 13:35:18.095704 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.099933 (+ 4229us) service_pool.cc:227] Handling call
0808 13:35:20.473532 (+2373599us) ts_tablet_manager.cc:1145] Deleting tablet 81fb3e099f7f4544b3c100070f0d14ef
0808 13:35:20.473572 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.525325 (+ 51753us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.547564 (+ 22239us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.550508 (+ 2944us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2372328}
I20260808 13:35:20.555876 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 81fb3e099f7f4544b3c100070f0d14ef (table default.Table_76c233d2deef48ec8ec2fba9d29f6ecb [id=bdb7331763214cbe82fe544204f8b2a3]) successfully deleted
I20260808 13:35:20.558948 6591 tablet_replica.cc:330] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:20.559646 6591 raft_consensus.cc:2229] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:20.560060 6591 raft_consensus.cc:2258] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:20.561614 6591 ts_tablet_manager.cc:1905] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.563397 6026 tablet_bootstrap.cc:492] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:20.563777 6026 ts_tablet_manager.cc:1397] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.041s user 0.005s sys 0.004s
I20260808 13:35:20.564162 6765 raft_consensus.cc:686] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:20.566929 6587 ts_tablet_manager.cc:1918] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:20.566779 6026 raft_consensus.cc:348] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.567293 6587 log.cc:1196] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/345d13cbde6844079c2d1eb80732bb1f
I20260808 13:35:20.567432 6026 raft_consensus.cc:374] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:20.567689 6026 raft_consensus.cc:729] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:20.568013 6587 ts_tablet_manager.cc:1939] T 345d13cbde6844079c2d1eb80732bb1f P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:20.566290 6765 consensus_queue.cc:237] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.572264 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 142) took 2775 ms. Trace:
I20260808 13:35:20.572554 6587 rpcz_store.cc:276] 0808 13:35:17.797144 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.803690 (+ 6546us) service_pool.cc:227] Handling call
0808 13:35:20.297855 (+2494165us) ts_tablet_manager.cc:1145] Deleting tablet 345d13cbde6844079c2d1eb80732bb1f
0808 13:35:20.306286 (+ 8431us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.453884 (+147598us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.566697 (+112813us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.572196 (+ 5499us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2492512}
I20260808 13:35:20.573725 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 345d13cbde6844079c2d1eb80732bb1f (table default.Table_7ad3b44b6be7476881db975f93533b24 [id=cc59d440766d4d2c983f64bca86e7d21]) successfully deleted
I20260808 13:35:20.584713 4840 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a5c2164ca2344a5fa5cf5b85ddf5dfe5" } new_table_name: "default.Table_ceeec8b3cee9414bad671191ab524f9d" modify_external_catalogs: true
I20260808 13:35:20.590395 4835 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet a56f8e4a166643698f558840ce8975f5
I20260808 13:35:20.595398 4835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_49299116c37c4b9dbb509dbc8efae85e" } modify_external_catalogs: true
I20260808 13:35:20.596108 4835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_49299116c37c4b9dbb509dbc8efae85e" } modify_external_catalogs: true
I20260808 13:35:20.571478 6026 consensus_queue.cc:260] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.596784 6591 ts_tablet_manager.cc:1918] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:20.603847 6591 log.cc:1196] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/d2ac4c366363468aa62b73e0f1e04532
I20260808 13:35:20.604486 6591 ts_tablet_manager.cc:1939] T d2ac4c366363468aa62b73e0f1e04532 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:20.604637 6026 ts_tablet_manager.cc:1428] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.041s user 0.008s sys 0.000s
I20260808 13:35:20.605810 6026 tablet_bootstrap.cc:492] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:20.607681 6587 tablet_replica.cc:330] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:20.608414 6587 raft_consensus.cc:2229] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:20.608871 6587 raft_consensus.cc:2258] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:20.610210 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 147) took 2514 ms. Trace:
I20260808 13:35:20.610499 6591 rpcz_store.cc:276] 0808 13:35:18.096011 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.101389 (+ 5378us) service_pool.cc:227] Handling call
0808 13:35:20.558807 (+2457418us) ts_tablet_manager.cc:1145] Deleting tablet d2ac4c366363468aa62b73e0f1e04532
0808 13:35:20.558843 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.589487 (+ 30644us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.596573 (+ 7086us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.610152 (+ 13579us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2455936}
I20260808 13:35:20.611680 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet d2ac4c366363468aa62b73e0f1e04532 (table default.Table_d81d55e146d84d41a68894ad0dd4dc93 [id=65666ad40b274ca9a277269157ccbb97]) successfully deleted
I20260808 13:35:20.615172 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 152) took 2015 ms. Trace:
I20260808 13:35:20.615545 6591 rpcz_store.cc:276] 0808 13:35:18.599768 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.599923 (+ 155us) service_pool.cc:227] Handling call
0808 13:35:20.614846 (+2014923us) ts_tablet_manager.cc:1145] Deleting tablet 264ee45f8dfa4a05b0e5e812ce1b622c
0808 13:35:20.614887 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.615119 (+ 232us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2013516}
W20260808 13:35:20.616808 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 264ee45f8dfa4a05b0e5e812ce1b622c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 264ee45f8dfa4a05b0e5e812ce1b622c already in progress: creating tablet
I20260808 13:35:20.617995 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 153) took 1982 ms. Trace:
I20260808 13:35:20.619421 6591 rpcz_store.cc:276] 0808 13:35:18.651360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.716725 (+ 65365us) service_pool.cc:227] Handling call
0808 13:35:20.617676 (+1900951us) ts_tablet_manager.cc:1145] Deleting tablet 8b57966a9b7b4673ab9f98720cfd456d
0808 13:35:20.617715 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.617946 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1900293}
W20260808 13:35:20.623085 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 8b57966a9b7b4673ab9f98720cfd456d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b57966a9b7b4673ab9f98720cfd456d already in progress: creating tablet
I20260808 13:35:20.624711 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 158) took 1529 ms. Trace:
I20260808 13:35:20.625056 6591 rpcz_store.cc:276] 0808 13:35:19.095698 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.095894 (+ 196us) service_pool.cc:227] Handling call
0808 13:35:20.624404 (+1528510us) ts_tablet_manager.cc:1145] Deleting tablet 3a27517560d64a3b8da2cf55980e18a8
0808 13:35:20.624445 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.624662 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1527140}
W20260808 13:35:20.626166 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 3a27517560d64a3b8da2cf55980e18a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a27517560d64a3b8da2cf55980e18a8 already in progress: creating tablet
I20260808 13:35:20.626993 6591 tablet_replica.cc:330] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
W20260808 13:35:20.628717 6450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260808 13:35:20.629565 6591 raft_consensus.cc:2229] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:20.630077 6591 raft_consensus.cc:2258] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:20.632169 6591 ts_tablet_manager.cc:1905] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:20.632695 6450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:35:20.633136 6450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:35:20.633985 6450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:35:20.635950 6033 tablet_bootstrap.cc:492] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:20.636402 6033 ts_tablet_manager.cc:1397] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.276s user 0.011s sys 0.000s
I20260808 13:35:20.640192 6587 ts_tablet_manager.cc:1905] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.643553 6833 raft_consensus.cc:482] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:20.643996 6833 raft_consensus.cc:504] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.645867 6833 leader_election.cc:288] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:20.648144 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4dc66771804e0cb8991da63eacba0c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:20.651679 5551 leader_election.cc:341] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:20.660220 6026 tablet_bootstrap.cc:654] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:20.679648 6591 ts_tablet_manager.cc:1918] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:20.680124 6591 log.cc:1196] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/6d8018bf8ff843d8818fbba9473acf47
I20260808 13:35:20.680969 6591 ts_tablet_manager.cc:1939] T 6d8018bf8ff843d8818fbba9473acf47 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:20.683386 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 159) took 1586 ms. Trace:
I20260808 13:35:20.683789 6591 rpcz_store.cc:276] 0808 13:35:19.097670 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.097832 (+ 162us) service_pool.cc:227] Handling call
0808 13:35:20.626859 (+1529027us) ts_tablet_manager.cc:1145] Deleting tablet 6d8018bf8ff843d8818fbba9473acf47
0808 13:35:20.626896 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.667197 (+ 40301us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.679441 (+ 12244us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.683329 (+ 3888us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1528157}
I20260808 13:35:20.685207 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]) successfully deleted
I20260808 13:35:20.686502 6591 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 161) took 1086 ms. Trace:
I20260808 13:35:20.686888 6591 rpcz_store.cc:276] 0808 13:35:19.600416 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.600578 (+ 162us) service_pool.cc:227] Handling call
0808 13:35:20.686142 (+1085564us) ts_tablet_manager.cc:1145] Deleting tablet e992a0294c404cfb9541b57782a05fc9
0808 13:35:20.686184 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.686453 (+ 269us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1084943}
W20260808 13:35:20.688406 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet e992a0294c404cfb9541b57782a05fc9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e992a0294c404cfb9541b57782a05fc9 already in progress: creating tablet
W20260808 13:35:20.690380 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 9423b380a62c467895e7fdff14756586 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9423b380a62c467895e7fdff14756586 already in progress: creating tablet
W20260808 13:35:20.691956 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet d3682c9c656c4556bc13ae3aeddd7c5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3682c9c656c4556bc13ae3aeddd7c5e already in progress: creating tablet
I20260808 13:35:20.693022 6591 tablet_replica.cc:330] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:20.696842 4842 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 1 got 0
I20260808 13:35:20.696609 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4dc66771804e0cb8991da63eacba0c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:20.697616 5878 raft_consensus.cc:2454] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:20.698107 6591 raft_consensus.cc:2229] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:20.698992 6591 raft_consensus.cc:2258] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:20.698925 4842 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:20.698921 6737 raft_consensus.cc:482] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:20.699513 6737 raft_consensus.cc:504] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.700105 4842 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:20.700801 6591 ts_tablet_manager.cc:1905] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.703696 5550 leader_election.cc:302] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:20.705487 6833 raft_consensus.cc:2781] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:20.705948 6833 raft_consensus.cc:482] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:20.706480 6833 raft_consensus.cc:3037] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:20.712652 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fb3e099f7f4544b3c100070f0d14ef" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:20.714004 5771 leader_election.cc:341] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:20.715483 5772 leader_election.cc:341] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:20.716346 5772 leader_election.cc:302] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:20.717937 6810 raft_consensus.cc:2726] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:20.727226 6737 leader_election.cc:288] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:20.735153 6839 tablet.cc:1722] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:20.738456 6796 raft_consensus.cc:482] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:20.738893 6796 raft_consensus.cc:504] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.740741 6796 leader_election.cc:288] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:20.742743 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "973a7afdcbf944aab8f11b3fd3995865" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:20.743327 5637 raft_consensus.cc:2454] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:20.744683 5285 leader_election.cc:302] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:20.746019 6796 raft_consensus.cc:2781] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:20.746585 6796 raft_consensus.cc:482] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:20.748766 6796 raft_consensus.cc:3037] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:20.749680 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "973a7afdcbf944aab8f11b3fd3995865" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:20.751888 5283 leader_election.cc:341] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:20.764923 6585 ts_tablet_manager.cc:1918] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:20.765587 6585 log.cc:1196] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/2fa77744bcfb4a5cace730fd9f9fd1ca
I20260808 13:35:20.767740 6033 raft_consensus.cc:348] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.768808 6033 raft_consensus.cc:374] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:20.769272 6033 raft_consensus.cc:729] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:20.764700 6587 ts_tablet_manager.cc:1918] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:20.770180 6587 log.cc:1196] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/5b6e6ce417cc45fd90f5858c30d37a47
I20260808 13:35:20.754308 6833 raft_consensus.cc:504] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.770980 6587 ts_tablet_manager.cc:1939] T 5b6e6ce417cc45fd90f5858c30d37a47 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:20.775478 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4dc66771804e0cb8991da63eacba0c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:20.776894 5551 leader_election.cc:341] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:20.777777 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 144) took 2969 ms. Trace:
I20260808 13:35:20.777930 6793 raft_consensus.cc:482] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:20.778092 6587 rpcz_store.cc:276] 0808 13:35:17.808455 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.810846 (+ 2391us) service_pool.cc:227] Handling call
0808 13:35:20.605790 (+2794944us) ts_tablet_manager.cc:1145] Deleting tablet 5b6e6ce417cc45fd90f5858c30d37a47
0808 13:35:20.605840 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.694695 (+ 88855us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.764454 (+ 69759us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.777720 (+ 13266us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2794133}
I20260808 13:35:20.778728 6793 raft_consensus.cc:504] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.758368 6796 raft_consensus.cc:504] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.787339 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "973a7afdcbf944aab8f11b3fd3995865" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:20.788030 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 5b6e6ce417cc45fd90f5858c30d37a47 (table default.Table_ade5bcdd6da94c6284819d3fe37f039f [id=3d8a7488d67c47e291b817ee36e99ac0]) successfully deleted
I20260808 13:35:20.788239 5642 raft_consensus.cc:3037] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:20.791901 5772 consensus_peers.cc:487] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d8018bf8ff843d8818fbba9473acf47. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:20.793386 6796 leader_election.cc:288] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:20.797735 5285 leader_election.cc:341] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: d2ac4c366363468aa62b73e0f1e04532
I20260808 13:35:20.799162 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 143) took 2992 ms. Trace:
I20260808 13:35:20.799497 6587 rpcz_store.cc:276] 0808 13:35:17.806717 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.807906 (+ 1189us) service_pool.cc:227] Handling call
0808 13:35:20.798893 (+2990987us) ts_tablet_manager.cc:1145] Deleting tablet 8948ce98e7a941f59fa9932fe241638d
0808 13:35:20.798937 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.799104 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2982403}
W20260808 13:35:20.802594 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 8948ce98e7a941f59fa9932fe241638d because the tablet was not found. No further retry: Not found: Tablet not found: 8948ce98e7a941f59fa9932fe241638d
I20260808 13:35:20.807428 5386 raft_consensus.cc:3037] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:20.811671 6587 tablet_replica.cc:330] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:20.812064 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_1c113a710bb14973be6ef54f41e550f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:20.812507 6587 raft_consensus.cc:2229] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:20.812980 6587 raft_consensus.cc:2258] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:20.813740 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c113a710bb14973be6ef54f41e550f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:20.817718 6591 ts_tablet_manager.cc:1918] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:20.818141 6591 log.cc:1196] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/e7616d81aa924933a9b24d44004725fa
I20260808 13:35:20.818938 6591 ts_tablet_manager.cc:1939] T e7616d81aa924933a9b24d44004725fa P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:20.821229 6591 tablet_replica.cc:330] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:20.821964 6591 raft_consensus.cc:2229] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:20.823619 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet e7616d81aa924933a9b24d44004725fa (table default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9]) successfully deleted
I20260808 13:35:20.824050 6450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.4.170.62:34069,127.4.170.61:46555,127.4.170.60:37135
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.60:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.170.60
--webserver_port=45565
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:35:20.813943 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "973a7afdcbf944aab8f11b3fd3995865" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:20.825759 6450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:35:20.827154 6793 leader_election.cc:288] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:20.826455 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ac4c366363468aa62b73e0f1e04532" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:20.828068 6450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 13:35:20.822433 6591 raft_consensus.cc:2258] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:20.834286 5283 leader_election.cc:341] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:20.841459 6591 ts_tablet_manager.cc:1905] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.841514 6587 ts_tablet_manager.cc:1905] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:20.853520 6615 consensus_queue.cc:1035] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:20.858754 6033 consensus_queue.cc:260] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.878690 5878 raft_consensus.cc:2454] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:20.880076 6033 ts_tablet_manager.cc:1428] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.229s user 0.008s sys 0.000s
I20260808 13:35:20.880962 6033 tablet_bootstrap.cc:492] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:20.881632 5283 leader_election.cc:302] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:20.863459 6585 ts_tablet_manager.cc:1939] T 2fa77744bcfb4a5cace730fd9f9fd1ca P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
W20260808 13:35:20.882541 5283 raft_consensus.cc:2670] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260808 13:35:20.885128 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 128) took 4070 ms. Trace:
I20260808 13:35:20.885568 6585 rpcz_store.cc:276] 0808 13:35:16.836864 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.844944 (+ 8080us) service_pool.cc:227] Handling call
0808 13:35:20.263488 (+3418544us) ts_tablet_manager.cc:1145] Deleting tablet 2fa77744bcfb4a5cace730fd9f9fd1ca
0808 13:35:20.263584 (+ 96us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.522039 (+258455us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.764612 (+242573us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.885056 (+120444us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3340205}
I20260808 13:35:20.886142 6046 tablet_bootstrap.cc:492] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:20.886720 6046 ts_tablet_manager.cc:1397] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.476s user 0.105s sys 0.156s
I20260808 13:35:20.891422 6591 ts_tablet_manager.cc:1918] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:20.891372 6046 raft_consensus.cc:348] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.891784 6591 log.cc:1196] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/543debff6da54d26bf62260bf8721ff7
I20260808 13:35:20.892009 6046 raft_consensus.cc:374] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:20.892441 6591 ts_tablet_manager.cc:1939] T 543debff6da54d26bf62260bf8721ff7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:20.893914 6793 raft_consensus.cc:482] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:20.894426 6793 raft_consensus.cc:504] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.899734 6793 leader_election.cc:288] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:20.900890 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2173e97590be43f9a24f55689d796269" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:20.926805 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 69642fdba1f145438b04eae9f8154101
I20260808 13:35:20.928455 4839 catalog_manager.cc:6565] ReplaceTablet: tablet 69642fdba1f145438b04eae9f8154101 deleted and replaced by tablet 538d9a883e9d46668fde30876352d20e
I20260808 13:35:20.929385 4839 master_service.cc:946] ReplaceTablet: received request to replace tablet 5b6e6ce417cc45fd90f5858c30d37a47 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:20.932265 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 69642fdba1f145438b04eae9f8154101 with delete_type TABLET_DATA_DELETED (replaced by tablet 538d9a883e9d46668fde30876352d20e) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:20.932328 6853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:35:20.892259 6046 raft_consensus.cc:729] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:20.935402 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 69642fdba1f145438b04eae9f8154101 with delete_type TABLET_DATA_DELETED (replaced by tablet 538d9a883e9d46668fde30876352d20e) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:20.939682 6046 consensus_queue.cc:260] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:20.942709 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c3cd224a54354169a0cba0ad45c3257e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:20.945581 6046 ts_tablet_manager.cc:1428] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.058s user 0.003s sys 0.002s
I20260808 13:35:20.951800 6587 ts_tablet_manager.cc:1918] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:20.952205 6587 log.cc:1196] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/d2ac4c366363468aa62b73e0f1e04532
I20260808 13:35:20.952888 6587 ts_tablet_manager.cc:1939] T d2ac4c366363468aa62b73e0f1e04532 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:20.954459 6046 tablet_bootstrap.cc:492] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:20.965447 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c3cd224a54354169a0cba0ad45c3257e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:20.974061 6046 tablet_bootstrap.cc:654] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:20.976637 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 146) took 3134 ms. Trace:
I20260808 13:35:20.978206 6615 consensus_queue.cc:1035] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:20.977998 6587 rpcz_store.cc:276] 0808 13:35:17.842465 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.845440 (+ 2975us) service_pool.cc:227] Handling call
0808 13:35:20.811504 (+2966064us) ts_tablet_manager.cc:1145] Deleting tablet d2ac4c366363468aa62b73e0f1e04532
0808 13:35:20.811551 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.904272 (+ 92721us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.951567 (+ 47295us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:20.976560 (+ 24993us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2925074}
I20260808 13:35:20.982868 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 2fa77744bcfb4a5cace730fd9f9fd1ca (table default.Table_1f0431ee90fa4e2193747a84078f912c [id=c16baabb989244459aec3f7506aaa959]) successfully deleted
I20260808 13:35:20.983271 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet d2ac4c366363468aa62b73e0f1e04532 (table default.Table_d81d55e146d84d41a68894ad0dd4dc93 [id=65666ad40b274ca9a277269157ccbb97]) successfully deleted
I20260808 13:35:20.984285 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 543debff6da54d26bf62260bf8721ff7 (table default.Table_062151f3d1444efc8a42ba6cd06e19c9 [id=68dee345bd2f4fc1823048b3da8d944f]) successfully deleted
I20260808 13:35:20.984501 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 133) took 3957 ms. Trace:
I20260808 13:35:20.985119 6585 rpcz_store.cc:276] 0808 13:35:17.027081 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.027233 (+ 152us) service_pool.cc:227] Handling call
0808 13:35:20.984110 (+3956877us) ts_tablet_manager.cc:1145] Deleting tablet 127e8d0323954ef788018bcb91582951
0808 13:35:20.984158 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.984438 (+ 280us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3956113}
W20260808 13:35:20.986586 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 127e8d0323954ef788018bcb91582951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 127e8d0323954ef788018bcb91582951 already in progress: creating tablet
I20260808 13:35:20.988998 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 134) took 3932 ms. Trace:
I20260808 13:35:20.989460 6585 rpcz_store.cc:276] 0808 13:35:17.056715 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.056888 (+ 173us) service_pool.cc:227] Handling call
0808 13:35:20.987399 (+3930511us) ts_tablet_manager.cc:1145] Deleting tablet fc9b2e69fe27431dac9eb5ecb9e3e23e
0808 13:35:20.987443 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:20.988933 (+ 1490us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3929663}
W20260808 13:35:20.994481 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 74980.7us
W20260808 13:35:21.014514 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet cfc9a6ec9a86431fb46e50fcaa658d33 because the tablet was not found. No further retry: Not found: Tablet not found: cfc9a6ec9a86431fb46e50fcaa658d33
I20260808 13:35:21.015486 6026 tablet_bootstrap.cc:492] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:21.015897 6026 ts_tablet_manager.cc:1397] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.410s user 0.011s sys 0.006s
W20260808 13:35:21.016863 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 851a0b374a854efd87dd66e39b9df2d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 851a0b374a854efd87dd66e39b9df2d5 already in progress: creating tablet
I20260808 13:35:20.982463 5642 raft_consensus.cc:2454] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:21.018487 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 69642fdba1f145438b04eae9f8154101 with delete_type TABLET_DATA_DELETED (replaced by tablet 538d9a883e9d46668fde30876352d20e) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:21.015936 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 145) took 3207 ms. Trace:
I20260808 13:35:21.021061 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2173e97590be43f9a24f55689d796269" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:21.021946 6833 leader_election.cc:288] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:21.022739 5285 leader_election.cc:302] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:21.023262 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 69642fdba1f145438b04eae9f8154101 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69642fdba1f145438b04eae9f8154101 already in progress: creating tablet
I20260808 13:35:21.024291 6793 raft_consensus.cc:2781] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:21.024781 6793 raft_consensus.cc:686] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:21.025624 6793 consensus_queue.cc:237] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.017976 6026 raft_consensus.cc:348] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.030061 6026 raft_consensus.cc:374] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:21.031546 6026 raft_consensus.cc:729] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:21.033034 6026 consensus_queue.cc:260] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.038981 6026 ts_tablet_manager.cc:1428] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.023s user 0.006s sys 0.001s
I20260808 13:35:21.042127 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_7d31aaee9dd04cabbbbca675bde3a2f4" } modify_external_catalogs: true
I20260808 13:35:21.042778 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_7d31aaee9dd04cabbbbca675bde3a2f4" } modify_external_catalogs: true
I20260808 13:35:21.019657 6587 rpcz_store.cc:276] 0808 13:35:17.808859 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.838072 (+ 29213us) service_pool.cc:227] Handling call
0808 13:35:21.015539 (+3177467us) ts_tablet_manager.cc:1145] Deleting tablet 81fb3e099f7f4544b3c100070f0d14ef
0808 13:35:21.015593 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.015872 (+ 279us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3112893}
W20260808 13:35:21.053161 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 81fb3e099f7f4544b3c100070f0d14ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81fb3e099f7f4544b3c100070f0d14ef already in progress: creating tablet
I20260808 13:35:21.053490 6026 tablet_bootstrap.cc:492] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:21.054319 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 152) took 2446 ms. Trace:
I20260808 13:35:21.054747 6587 rpcz_store.cc:276] 0808 13:35:18.608074 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.643446 (+ 35372us) service_pool.cc:227] Handling call
0808 13:35:21.053927 (+2410481us) ts_tablet_manager.cc:1145] Deleting tablet 264ee45f8dfa4a05b0e5e812ce1b622c
0808 13:35:21.053975 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.054253 (+ 278us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2289455}
I20260808 13:35:21.056269 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 153) took 2439 ms. Trace:
W20260808 13:35:21.057752 4840 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4b7284afe0294644a368cf51451c7ab6 consensus state has no leader
I20260808 13:35:21.060657 5285 leader_election.cc:302] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:21.062451 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 264ee45f8dfa4a05b0e5e812ce1b622c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 264ee45f8dfa4a05b0e5e812ce1b622c already in progress: creating tablet
I20260808 13:35:21.065503 6046 tablet_bootstrap.cc:492] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:21.065936 6046 ts_tablet_manager.cc:1397] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.112s user 0.010s sys 0.000s
I20260808 13:35:21.056579 6587 rpcz_store.cc:276] 0808 13:35:18.616871 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.631276 (+ 14405us) service_pool.cc:227] Handling call
0808 13:35:21.055928 (+2424652us) ts_tablet_manager.cc:1145] Deleting tablet 8b57966a9b7b4673ab9f98720cfd456d
0808 13:35:21.055971 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.056218 (+ 247us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2281005}
W20260808 13:35:21.075201 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 8b57966a9b7b4673ab9f98720cfd456d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b57966a9b7b4673ab9f98720cfd456d already in progress: creating tablet
I20260808 13:35:21.077812 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 159) took 2014 ms. Trace:
I20260808 13:35:21.079852 6046 raft_consensus.cc:348] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.080696 6046 raft_consensus.cc:374] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:21.080978 6046 raft_consensus.cc:729] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:21.078163 6587 rpcz_store.cc:276] 0808 13:35:19.063564 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.064147 (+ 583us) service_pool.cc:227] Handling call
0808 13:35:21.077445 (+2013298us) ts_tablet_manager.cc:1145] Deleting tablet 3a27517560d64a3b8da2cf55980e18a8
0808 13:35:21.077490 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.077753 (+ 263us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2012679}
I20260808 13:35:21.084092 6026 tablet_bootstrap.cc:654] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:21.090149 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 3a27517560d64a3b8da2cf55980e18a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a27517560d64a3b8da2cf55980e18a8 already in progress: creating tablet
W20260808 13:35:20.995617 5772 consensus_peers.cc:487] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e2a95712de254c13b3b6dc4fe5cb15f7. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:21.021479 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4dc66771804e0cb8991da63eacba0c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:21.090840 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 95339.8us
W20260808 13:35:21.091526 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc9b2e69fe27431dac9eb5ecb9e3e23e already in progress: creating tablet
I20260808 13:35:21.091639 6585 tablet_replica.cc:330] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:21.092588 6587 tablet_replica.cc:330] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:21.093395 6587 raft_consensus.cc:2229] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:21.093897 6587 raft_consensus.cc:2258] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:21.090991 5882 raft_consensus.cc:3037] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:21.101998 6033 tablet_bootstrap.cc:654] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:21.105420 6585 raft_consensus.cc:2229] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:21.105909 6585 raft_consensus.cc:2258] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:21.111807 6585 ts_tablet_manager.cc:1905] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:21.119172 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_2d152c1c176a4bc58ff1f37b2a9bfd0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:21.120298 5882 raft_consensus.cc:2454] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
W20260808 13:35:21.120827 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d152c1c176a4bc58ff1f37b2a9bfd0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:21.136006 5550 leader_election.cc:302] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:21.137190 6833 raft_consensus.cc:2781] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:21.137617 6833 raft_consensus.cc:686] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:21.138381 6833 consensus_queue.cc:237] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.148947 6793 raft_consensus.cc:2726] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:21.162213 6026 tablet_bootstrap.cc:492] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:21.163727 6026 ts_tablet_manager.cc:1397] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.111s user 0.009s sys 0.003s
I20260808 13:35:21.169831 6046 consensus_queue.cc:260] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.172950 6046 ts_tablet_manager.cc:1428] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.107s user 0.007s sys 0.000s
I20260808 13:35:21.183756 6587 ts_tablet_manager.cc:1905] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:21.190104 6033 tablet_bootstrap.cc:492] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:21.190503 6033 ts_tablet_manager.cc:1397] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.310s user 0.009s sys 0.001s
I20260808 13:35:21.197854 6046 tablet_bootstrap.cc:492] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:21.195753 6585 ts_tablet_manager.cc:1918] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:21.202222 6585 log.cc:1196] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/a0b29786cb404bc2b01473c3b8390a50
I20260808 13:35:21.203228 6585 ts_tablet_manager.cc:1939] T a0b29786cb404bc2b01473c3b8390a50 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:21.206993 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 135) took 4040 ms. Trace:
I20260808 13:35:21.207300 6585 rpcz_store.cc:276] 0808 13:35:17.166808 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.166954 (+ 146us) service_pool.cc:227] Handling call
0808 13:35:21.091470 (+3924516us) ts_tablet_manager.cc:1145] Deleting tablet a0b29786cb404bc2b01473c3b8390a50
0808 13:35:21.091516 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.139304 (+ 47788us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.195514 (+ 56210us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.206937 (+ 11423us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3923730}
I20260808 13:35:21.208559 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet a0b29786cb404bc2b01473c3b8390a50 (table default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]) successfully deleted
I20260808 13:35:21.237532 6026 raft_consensus.cc:348] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.238253 6026 raft_consensus.cc:374] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:21.197022 6033 raft_consensus.cc:348] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.242533 6033 raft_consensus.cc:374] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:21.244117 6033 raft_consensus.cc:729] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:21.238543 6026 raft_consensus.cc:729] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:21.255664 6026 consensus_queue.cc:260] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.260648 6585 tablet_replica.cc:330] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:21.262928 6033 consensus_queue.cc:260] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.264679 6585 raft_consensus.cc:2229] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:21.268100 6585 raft_consensus.cc:2258] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:21.295151 6026 ts_tablet_manager.cc:1428] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.129s user 0.006s sys 0.004s
I20260808 13:35:21.301697 6026 tablet_bootstrap.cc:492] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:21.273936 5772 consensus_peers.cc:487] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 543debff6da54d26bf62260bf8721ff7. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:21.316916 6585 ts_tablet_manager.cc:1905] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:21.318240 6033 ts_tablet_manager.cc:1428] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.127s user 0.006s sys 0.002s
I20260808 13:35:21.350227 6033 tablet_bootstrap.cc:492] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:21.358022 6046 tablet_bootstrap.cc:654] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:21.364981 6026 tablet_bootstrap.cc:654] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:21.411681 6585 ts_tablet_manager.cc:1918] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260808 13:35:21.417495 4823 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be (table default.Table_d553a5c837764412ab01800bea0017c2 [id=81ee34de9d5c494fb2bd74557a174c14], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be consensus state has no leader
I20260808 13:35:21.420440 6026 tablet_bootstrap.cc:492] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:21.419945 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff163c36c6d42238cce2396102f9a68 (DEFAULT_TABLE table=default.Table_00b62ebc438b40209bd662e8cd04fb29 [id=e90bcbe52e7b4ad1b60f0324fe158df4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.421023 6026 ts_tablet_manager.cc:1397] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.120s user 0.009s sys 0.004s
I20260808 13:35:21.420449 5337 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff163c36c6d42238cce2396102f9a68 (DEFAULT_TABLE table=default.Table_00b62ebc438b40209bd662e8cd04fb29 [id=e90bcbe52e7b4ad1b60f0324fe158df4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.421905 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff163c36c6d42238cce2396102f9a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.422048 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff163c36c6d42238cce2396102f9a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.422652 6585 log.cc:1196] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/e2a95712de254c13b3b6dc4fe5cb15f7
I20260808 13:35:21.423444 6585 ts_tablet_manager.cc:1939] T e2a95712de254c13b3b6dc4fe5cb15f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:21.424116 6026 raft_consensus.cc:348] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:21.424768 6026 raft_consensus.cc:374] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:21.425028 6026 raft_consensus.cc:729] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:21.425418 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 136) took 4198 ms. Trace:
I20260808 13:35:21.425715 6585 rpcz_store.cc:276] 0808 13:35:17.227158 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.228332 (+ 1174us) service_pool.cc:227] Handling call
0808 13:35:21.260491 (+4032159us) ts_tablet_manager.cc:1145] Deleting tablet e2a95712de254c13b3b6dc4fe5cb15f7
0808 13:35:21.260534 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.376791 (+116257us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.411449 (+ 34658us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.425357 (+ 13908us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4031188}
I20260808 13:35:21.428344 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 1 got 0
I20260808 13:35:21.429651 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:21.433836 5353 tablet_service.cc:1467] Processing CreateTablet for tablet a73e98a8b20144508a5a8d451f4bb302 (DEFAULT_TABLE table=default.Table_c7d6c5128772436194a73fb2f48105b4 [id=12c0608843fa49cb9c2c931862415447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.434832 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a73e98a8b20144508a5a8d451f4bb302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.439896 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 138) took 3966 ms. Trace:
I20260808 13:35:21.440205 6585 rpcz_store.cc:276] 0808 13:35:17.474016 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.492046 (+ 18030us) service_pool.cc:227] Handling call
0808 13:35:21.426625 (+3934579us) ts_tablet_manager.cc:1145] Deleting tablet e00f2e0b98af431d99894e8778257bc4
0808 13:35:21.426665 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.439830 (+ 13165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3928942}
I20260808 13:35:21.441274 6585 tablet_replica.cc:330] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:21.441025 5614 tablet_service.cc:1467] Processing CreateTablet for tablet a73e98a8b20144508a5a8d451f4bb302 (DEFAULT_TABLE table=default.Table_c7d6c5128772436194a73fb2f48105b4 [id=12c0608843fa49cb9c2c931862415447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.441995 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a73e98a8b20144508a5a8d451f4bb302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.442031 6585 raft_consensus.cc:2229] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:21.442492 6585 raft_consensus.cc:2258] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:21.443655 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 594afd4174644c218951b9df82360bfe (DEFAULT_TABLE table=default.Table_48ee76308d8846b49515e00b6bf24244 [id=11a917856b1045d1891af36f7a71e26c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.444715 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 594afd4174644c218951b9df82360bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.444566 6026 consensus_queue.cc:260] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:21.449352 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 594afd4174644c218951b9df82360bfe (DEFAULT_TABLE table=default.Table_48ee76308d8846b49515e00b6bf24244 [id=11a917856b1045d1891af36f7a71e26c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.450814 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 594afd4174644c218951b9df82360bfe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:21.363723 4784 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 88563.1us
I20260808 13:35:21.455010 6585 ts_tablet_manager.cc:1905] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:21.472460 6026 ts_tablet_manager.cc:1428] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.051s user 0.008s sys 0.000s
I20260808 13:35:21.480723 6585 ts_tablet_manager.cc:1918] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:21.482547 5338 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa86f45770d44e7b993b240607ef2fc (DEFAULT_TABLE table=default.Table_b9063cf92547431f9c63a652c79fbaae [id=53aad8e2f83c4f8d87add3777f909fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.483071 6585 log.cc:1196] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:21.484308 5338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa86f45770d44e7b993b240607ef2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.486829 6585 ts_tablet_manager.cc:1939] T 81fb3e099f7f4544b3c100070f0d14ef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:21.494184 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 150) took 3709 ms. Trace:
I20260808 13:35:21.494513 6585 rpcz_store.cc:276] 0808 13:35:17.784712 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.801806 (+ 17094us) service_pool.cc:227] Handling call
0808 13:35:21.441120 (+3639314us) ts_tablet_manager.cc:1145] Deleting tablet 81fb3e099f7f4544b3c100070f0d14ef
0808 13:35:21.441160 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.462003 (+ 20843us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.480520 (+ 18517us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.494123 (+ 13603us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3638592}
I20260808 13:35:21.495724 6585 tablet_replica.cc:330] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:21.495687 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 1 got 0
I20260808 13:35:21.496730 6585 raft_consensus.cc:2229] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:21.497161 6585 raft_consensus.cc:2258] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:21.497128 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_bb876d9598e24abf96366e835a8b2d59 [id=16cef8437603476b97a8562edba58a41]). Expected version 2 got 0
I20260808 13:35:21.497929 6026 tablet_bootstrap.cc:492] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:21.498037 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:21.498804 6585 ts_tablet_manager.cc:1905] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:21.511446 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa86f45770d44e7b993b240607ef2fc (DEFAULT_TABLE table=default.Table_b9063cf92547431f9c63a652c79fbaae [id=53aad8e2f83c4f8d87add3777f909fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.512658 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa86f45770d44e7b993b240607ef2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.515368 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 249dcaf769934b75945d79d827a96be6 (DEFAULT_TABLE table=default.Table_0ae56405f82f446dbd5ed65b0062fc62 [id=6a9d5e1c30ff4f5490feb1eb08805fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.515564 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 249dcaf769934b75945d79d827a96be6 (DEFAULT_TABLE table=default.Table_0ae56405f82f446dbd5ed65b0062fc62 [id=6a9d5e1c30ff4f5490feb1eb08805fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.516371 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 249dcaf769934b75945d79d827a96be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.516480 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 249dcaf769934b75945d79d827a96be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.534235 6587 ts_tablet_manager.cc:1918] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:21.537606 6026 tablet_bootstrap.cc:654] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:21.559357 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet e00f2e0b98af431d99894e8778257bc4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e00f2e0b98af431d99894e8778257bc4 already in progress: creating tablet
I20260808 13:35:21.562394 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 81fb3e099f7f4544b3c100070f0d14ef (table default.Table_76c233d2deef48ec8ec2fba9d29f6ecb [id=bdb7331763214cbe82fe544204f8b2a3]) successfully deleted
I20260808 13:35:21.566219 4835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 1f9bc338093f403881d329a7e913c9f7
I20260808 13:35:21.567615 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 594afd4174644c218951b9df82360bfe (DEFAULT_TABLE table=default.Table_48ee76308d8846b49515e00b6bf24244 [id=11a917856b1045d1891af36f7a71e26c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.568785 5834 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff163c36c6d42238cce2396102f9a68 (DEFAULT_TABLE table=default.Table_00b62ebc438b40209bd662e8cd04fb29 [id=e90bcbe52e7b4ad1b60f0324fe158df4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.569936 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 594afd4174644c218951b9df82360bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.567615 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa86f45770d44e7b993b240607ef2fc (DEFAULT_TABLE table=default.Table_b9063cf92547431f9c63a652c79fbaae [id=53aad8e2f83c4f8d87add3777f909fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.571158 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9bc338093f403881d329a7e913c9f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:20 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:21.571401 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9bc338093f403881d329a7e913c9f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:20 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:21.571954 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa86f45770d44e7b993b240607ef2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.572742 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff163c36c6d42238cce2396102f9a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.580343 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 369a942248bd4cec9ee9170fe55601a3
I20260808 13:35:21.584745 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 369a942248bd4cec9ee9170fe55601a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:21.591331 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9bc338093f403881d329a7e913c9f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:20 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:21.591218 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 369a942248bd4cec9ee9170fe55601a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:21.593236 5845 tablet_service.cc:1467] Processing CreateTablet for tablet a73e98a8b20144508a5a8d451f4bb302 (DEFAULT_TABLE table=default.Table_c7d6c5128772436194a73fb2f48105b4 [id=12c0608843fa49cb9c2c931862415447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.594383 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a73e98a8b20144508a5a8d451f4bb302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.595088 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 86c2b34c95244472beec43643e60ff5f (DEFAULT_TABLE table=default.Table_2df92438979447cc89b7a5301a13d6a9 [id=40cafe491872437cba9d7294fff5519a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.596858 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86c2b34c95244472beec43643e60ff5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.596633 5600 tablet_service.cc:1467] Processing CreateTablet for tablet 86c2b34c95244472beec43643e60ff5f (DEFAULT_TABLE table=default.Table_2df92438979447cc89b7a5301a13d6a9 [id=40cafe491872437cba9d7294fff5519a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.598060 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86c2b34c95244472beec43643e60ff5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.598534 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 249dcaf769934b75945d79d827a96be6 (DEFAULT_TABLE table=default.Table_0ae56405f82f446dbd5ed65b0062fc62 [id=6a9d5e1c30ff4f5490feb1eb08805fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.599761 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 249dcaf769934b75945d79d827a96be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.606091 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 86c2b34c95244472beec43643e60ff5f (DEFAULT_TABLE table=default.Table_2df92438979447cc89b7a5301a13d6a9 [id=40cafe491872437cba9d7294fff5519a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.607221 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86c2b34c95244472beec43643e60ff5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.610991 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 369a942248bd4cec9ee9170fe55601a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:21.622145 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 1 got 0
I20260808 13:35:21.623843 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:21.534593 6587 log.cc:1196] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/6d8018bf8ff843d8818fbba9473acf47
I20260808 13:35:21.627861 6587 ts_tablet_manager.cc:1939] T 6d8018bf8ff843d8818fbba9473acf47 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:21.628621 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_55de3cd70f784a7c90b1028236c8118e" } new_table_name: "default.Table_114737be7a5c48089f667a5f02f76b5e" modify_external_catalogs: true
I20260808 13:35:21.629796 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 160) took 2565 ms. Trace:
I20260808 13:35:21.631385 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a16be245c9864b969707b18ee86278d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:21.633001 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a16be245c9864b969707b18ee86278d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:21.646754 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 1f9bc338093f403881d329a7e913c9f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f9bc338093f403881d329a7e913c9f7 already in progress: creating tablet
I20260808 13:35:21.648551 6033 tablet_bootstrap.cc:654] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:21.656682 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 2d8e4af746774747a70d425068e8dd63 (DEFAULT_TABLE table=default.Table_256bb136799848f4891c3923692e5feb [id=95f59bd3447c44c291270c448f53d78b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.657699 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d8e4af746774747a70d425068e8dd63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.657337 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 2d8e4af746774747a70d425068e8dd63 (DEFAULT_TABLE table=default.Table_256bb136799848f4891c3923692e5feb [id=95f59bd3447c44c291270c448f53d78b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.659368 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d8e4af746774747a70d425068e8dd63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.662583 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 2d8e4af746774747a70d425068e8dd63 (DEFAULT_TABLE table=default.Table_256bb136799848f4891c3923692e5feb [id=95f59bd3447c44c291270c448f53d78b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.667325 6046 tablet_bootstrap.cc:492] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:21.667721 6046 ts_tablet_manager.cc:1397] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.470s user 0.012s sys 0.004s
I20260808 13:35:21.668391 6026 tablet_bootstrap.cc:492] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:21.668833 6026 ts_tablet_manager.cc:1397] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.171s user 0.017s sys 0.011s
I20260808 13:35:21.669991 6046 raft_consensus.cc:348] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.671363 6026 raft_consensus.cc:348] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:21.672127 6026 raft_consensus.cc:374] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:21.672410 6026 raft_consensus.cc:729] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:21.673074 6026 consensus_queue.cc:260] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:21.674420 4833 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 538d9a883e9d46668fde30876352d20e has no consensus state
I20260808 13:35:21.675130 4833 master_service.cc:946] ReplaceTablet: received request to replace tablet 3a27517560d64a3b8da2cf55980e18a8 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:21.676957 4835 master_service.cc:946] ReplaceTablet: received request to replace tablet 60f8863bc32e49399940912444bcb972 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:21.678963 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d3498dfbc8c04e42abefc8e337816f9b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:21.680168 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d8e4af746774747a70d425068e8dd63. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:21.680768 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3498dfbc8c04e42abefc8e337816f9b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:21.687083 6026 ts_tablet_manager.cc:1428] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.018s user 0.007s sys 0.001s
I20260808 13:35:21.693783 5606 tablet_service.cc:1467] Processing CreateTablet for tablet d3b2cce690cd462aa58ceee2bd28c638 (DEFAULT_TABLE table=default.Table_206fd86bd0f44e8e9dc38631edff49ca [id=1a31b0d0c87b446cb7662a856510b67e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.699805 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b2cce690cd462aa58ceee2bd28c638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.699858 6026 tablet_bootstrap.cc:492] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:21.711416 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 64888f0d27c94ee7b25c0dda57e766f6 (DEFAULT_TABLE table=default.Table_93429697aa7c4405b439864c62c45c29 [id=feca1558e3e74fb2a9de982f5a2a5302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.713275 6026 tablet_bootstrap.cc:654] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:21.719077 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64888f0d27c94ee7b25c0dda57e766f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.720621 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be (DEFAULT_TABLE table=default.Table_d553a5c837764412ab01800bea0017c2 [id=81ee34de9d5c494fb2bd74557a174c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.722868 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.734233 5610 tablet_service.cc:1467] Processing CreateTablet for tablet a04d3dd028da4e7f86689e819548d0c5 (DEFAULT_TABLE table=default.Table_f685324c68e647af9888454398dc7d23 [id=01d0f24951d44f7f86fb55decd5990ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.739835 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a04d3dd028da4e7f86689e819548d0c5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:21.741598 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 369a942248bd4cec9ee9170fe55601a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 369a942248bd4cec9ee9170fe55601a3 already in progress: creating tablet
I20260808 13:35:21.740947 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c0f49fa8313743fbb5ea5d76f5262610"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:21.749121 6026 tablet_bootstrap.cc:492] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:21.749552 6026 ts_tablet_manager.cc:1397] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.050s user 0.010s sys 0.003s
I20260808 13:35:21.751922 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 1 got 0
I20260808 13:35:21.752849 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 973a7afdcbf944aab8f11b3fd3995865 (table default.Table_5c71e9adaaf5477b996f88813566c561 [id=eb20921cee2f48dc86300ce8ce5f52a1]). Expected version 3 got 0
I20260808 13:35:21.753976 6833 raft_consensus.cc:482] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:21.754555 6833 raft_consensus.cc:504] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:21.755131 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c0f49fa8313743fbb5ea5d76f5262610 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:21.754213 4838 catalog_manager.cc:5495] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:21.761543 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:21.688122 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 64888f0d27c94ee7b25c0dda57e766f6 (DEFAULT_TABLE table=default.Table_93429697aa7c4405b439864c62c45c29 [id=feca1558e3e74fb2a9de982f5a2a5302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.769291 6046 raft_consensus.cc:374] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:21.769582 6046 raft_consensus.cc:729] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:21.769979 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64888f0d27c94ee7b25c0dda57e766f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.690685 5340 tablet_service.cc:1467] Processing CreateTablet for tablet d3b2cce690cd462aa58ceee2bd28c638 (DEFAULT_TABLE table=default.Table_206fd86bd0f44e8e9dc38631edff49ca [id=1a31b0d0c87b446cb7662a856510b67e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.775384 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b2cce690cd462aa58ceee2bd28c638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.777076 4835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 60f8863bc32e49399940912444bcb972
I20260808 13:35:21.698443 5346 tablet_service.cc:1467] Processing CreateTablet for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be (DEFAULT_TABLE table=default.Table_d553a5c837764412ab01800bea0017c2 [id=81ee34de9d5c494fb2bd74557a174c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.630084 6587 rpcz_store.cc:276] 0808 13:35:19.063926 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.064070 (+ 144us) service_pool.cc:227] Handling call
0808 13:35:21.092421 (+2028351us) ts_tablet_manager.cc:1145] Deleting tablet 6d8018bf8ff843d8818fbba9473acf47
0808 13:35:21.092464 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.338319 (+245855us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.534041 (+195722us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.629734 (+ 95693us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2021243}
I20260808 13:35:21.741760 6793 raft_consensus.cc:482] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:21.778903 6793 raft_consensus.cc:504] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.779712 4835 catalog_manager.cc:6565] ReplaceTablet: tablet 60f8863bc32e49399940912444bcb972 deleted and replaced by tablet 62d68ab73bae4dd9af46e907b1fd198c
I20260808 13:35:21.780237 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f8863bc32e49399940912444bcb972 with delete_type TABLET_DATA_DELETED (replaced by tablet 62d68ab73bae4dd9af46e907b1fd198c) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:21.779332 5346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.780922 6793 leader_election.cc:288] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:21.780642 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_322ce0f117834a81978e588b16196c2a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:21.782085 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_322ce0f117834a81978e588b16196c2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:21.782224 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 161) took 2184 ms. Trace:
I20260808 13:35:21.783007 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]) successfully deleted
I20260808 13:35:21.782480 6587 rpcz_store.cc:276] 0808 13:35:19.598333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.598475 (+ 142us) service_pool.cc:227] Handling call
0808 13:35:21.781896 (+2183421us) ts_tablet_manager.cc:1145] Deleting tablet e992a0294c404cfb9541b57782a05fc9
0808 13:35:21.781936 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.782173 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2182630}
W20260808 13:35:21.783242 5285 leader_election.cc:341] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:21.785264 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 163) took 1978 ms. Trace:
I20260808 13:35:21.785544 6587 rpcz_store.cc:276] 0808 13:35:19.807236 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.807400 (+ 164us) service_pool.cc:227] Handling call
0808 13:35:21.784965 (+1977565us) ts_tablet_manager.cc:1145] Deleting tablet d3682c9c656c4556bc13ae3aeddd7c5e
0808 13:35:21.785002 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.785216 (+ 214us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1977034}
I20260808 13:35:21.785974 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f8863bc32e49399940912444bcb972 with delete_type TABLET_DATA_DELETED (replaced by tablet 62d68ab73bae4dd9af46e907b1fd198c) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:21.786568 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet e992a0294c404cfb9541b57782a05fc9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e992a0294c404cfb9541b57782a05fc9 already in progress: creating tablet
I20260808 13:35:21.786770 6888 tablet_replica.cc:330] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
W20260808 13:35:21.787204 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet d3682c9c656c4556bc13ae3aeddd7c5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3682c9c656c4556bc13ae3aeddd7c5e already in progress: creating tablet
I20260808 13:35:21.787652 6888 raft_consensus.cc:2229] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:21.788359 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 162) took 1981 ms. Trace:
I20260808 13:35:21.788743 6587 rpcz_store.cc:276] 0808 13:35:19.806824 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.834643 (+ 27819us) service_pool.cc:227] Handling call
0808 13:35:21.787972 (+1953329us) ts_tablet_manager.cc:1145] Deleting tablet 9423b380a62c467895e7fdff14756586
0808 13:35:21.788022 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.788298 (+ 276us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1943624}
I20260808 13:35:21.770164 6046 consensus_queue.cc:260] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:21.790023 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 9423b380a62c467895e7fdff14756586 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9423b380a62c467895e7fdff14756586 already in progress: creating tablet
I20260808 13:35:21.790795 6587 tablet_replica.cc:330] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:21.745217 5356 tablet_service.cc:1467] Processing CreateTablet for tablet a04d3dd028da4e7f86689e819548d0c5 (DEFAULT_TABLE table=default.Table_f685324c68e647af9888454398dc7d23 [id=01d0f24951d44f7f86fb55decd5990ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.797520 6888 raft_consensus.cc:2258] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:21.799528 6888 ts_tablet_manager.cc:1905] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:21.805893 6026 raft_consensus.cc:348] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:21.810753 6026 raft_consensus.cc:374] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:21.811021 6026 raft_consensus.cc:729] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:21.811144 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a04d3dd028da4e7f86689e819548d0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.811683 6026 consensus_queue.cc:260] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:21.818275 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1680269ff16246c382d6bccf44f8d52e" } modify_external_catalogs: true
I20260808 13:35:21.818902 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1680269ff16246c382d6bccf44f8d52e" } modify_external_catalogs: true
I20260808 13:35:21.827749 6793 raft_consensus.cc:482] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:21.828200 6793 raft_consensus.cc:504] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:21.830880 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4dc66771804e0cb8991da63eacba0c" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:21.831995 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a26598a69146e1950e9227523fff13" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:21.833223 6793 leader_election.cc:288] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:21.833464 5551 leader_election.cc:341] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:21.834811 6833 leader_election.cc:288] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:21.837453 4840 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 1 got 0
I20260808 13:35:21.838310 6585 ts_tablet_manager.cc:1918] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:21.838795 6585 log.cc:1196] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/d2ac4c366363468aa62b73e0f1e04532
I20260808 13:35:21.839608 6585 ts_tablet_manager.cc:1939] T d2ac4c366363468aa62b73e0f1e04532 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:21.841856 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 151) took 4023 ms. Trace:
W20260808 13:35:21.845114 5771 consensus_peers.cc:487] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d8018bf8ff843d8818fbba9473acf47. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:21.851356 6046 ts_tablet_manager.cc:1428] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.183s user 0.006s sys 0.000s
I20260808 13:35:21.859146 6888 ts_tablet_manager.cc:1918] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:21.859547 6888 log.cc:1196] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/60f8863bc32e49399940912444bcb972
I20260808 13:35:21.860323 6888 ts_tablet_manager.cc:1939] T 60f8863bc32e49399940912444bcb972 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:21.863073 6587 raft_consensus.cc:2229] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:21.863651 6587 raft_consensus.cc:2258] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:21.865679 6587 ts_tablet_manager.cc:1905] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:21.865630 6833 consensus_queue.cc:1035] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:21.862799 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 60f8863bc32e49399940912444bcb972 (table default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]) successfully deleted
I20260808 13:35:21.869899 4840 catalog_manager.cc:5495] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:21.871253 4840 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 973a7afdcbf944aab8f11b3fd3995865 (table default.Table_5c71e9adaaf5477b996f88813566c561 [id=eb20921cee2f48dc86300ce8ce5f52a1]). Expected version 3 got 0
I20260808 13:35:21.873466 4840 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:21.880684 6026 ts_tablet_manager.cc:1428] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.131s user 0.010s sys 0.016s
I20260808 13:35:21.842132 6585 rpcz_store.cc:276] 0808 13:35:17.818402 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:17.818579 (+ 177us) service_pool.cc:227] Handling call
0808 13:35:21.495580 (+3677001us) ts_tablet_manager.cc:1145] Deleting tablet d2ac4c366363468aa62b73e0f1e04532
0808 13:35:21.495618 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.519701 (+ 24083us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.690487 (+170786us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.841797 (+151310us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3676336}
I20260808 13:35:21.881660 6026 tablet_bootstrap.cc:492] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:21.890307 6026 tablet_bootstrap.cc:654] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:21.896097 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4dc66771804e0cb8991da63eacba0c" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:21.897111 5880 raft_consensus.cc:2379] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:21.902552 5844 tablet_service.cc:1467] Processing CreateTablet for tablet d3b2cce690cd462aa58ceee2bd28c638 (DEFAULT_TABLE table=default.Table_206fd86bd0f44e8e9dc38631edff49ca [id=1a31b0d0c87b446cb7662a856510b67e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.902961 6046 tablet_bootstrap.cc:492] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:21.903565 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b2cce690cd462aa58ceee2bd28c638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.907449 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be (DEFAULT_TABLE table=default.Table_d553a5c837764412ab01800bea0017c2 [id=81ee34de9d5c494fb2bd74557a174c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.908382 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.911389 5843 tablet_service.cc:1467] Processing CreateTablet for tablet a04d3dd028da4e7f86689e819548d0c5 (DEFAULT_TABLE table=default.Table_f685324c68e647af9888454398dc7d23 [id=01d0f24951d44f7f86fb55decd5990ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.912326 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a04d3dd028da4e7f86689e819548d0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.915033 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f8863bc32e49399940912444bcb972 with delete_type TABLET_DATA_DELETED (replaced by tablet 62d68ab73bae4dd9af46e907b1fd198c) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:21.920836 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 64888f0d27c94ee7b25c0dda57e766f6 (DEFAULT_TABLE table=default.Table_93429697aa7c4405b439864c62c45c29 [id=feca1558e3e74fb2a9de982f5a2a5302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:21.921810 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64888f0d27c94ee7b25c0dda57e766f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:21.927642 5283 leader_election.cc:302] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:21.928790 5283 leader_election.cc:341] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: 81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:21.929148 5283 leader_election.cc:302] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:21.929979 6796 raft_consensus.cc:3037] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:21.932741 6793 consensus_queue.cc:1035] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:21.939121 6911 raft_consensus.cc:2726] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:21.950491 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a26598a69146e1950e9227523fff13" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:21.963923 6046 tablet_bootstrap.cc:654] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:21.983840 6026 tablet_bootstrap.cc:492] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:21.984267 6026 ts_tablet_manager.cc:1397] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.103s user 0.014s sys 0.017s
I20260808 13:35:21.993492 6587 ts_tablet_manager.cc:1918] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:21.993877 6587 log.cc:1196] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/e7616d81aa924933a9b24d44004725fa
I20260808 13:35:22.013600 6026 raft_consensus.cc:348] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.014328 6026 raft_consensus.cc:374] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:22.014595 6026 raft_consensus.cc:729] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:22.018451 6587 ts_tablet_manager.cc:1939] T e7616d81aa924933a9b24d44004725fa P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:22.022148 6796 raft_consensus.cc:2726] T cc4dc66771804e0cb8991da63eacba0c P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:22.023973 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 164) took 2062 ms. Trace:
I20260808 13:35:22.024365 6587 rpcz_store.cc:276] 0808 13:35:19.961751 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.964506 (+ 2755us) service_pool.cc:227] Handling call
0808 13:35:21.790645 (+1826139us) ts_tablet_manager.cc:1145] Deleting tablet e7616d81aa924933a9b24d44004725fa
0808 13:35:21.790698 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:21.939310 (+148612us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:21.993281 (+ 53971us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:22.023906 (+ 30625us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":71301,"tablet-delete.queue_time_us":1824936}
I20260808 13:35:22.025841 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 166) took 1909 ms. Trace:
I20260808 13:35:22.028570 6026 consensus_queue.cc:260] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.031304 6026 ts_tablet_manager.cc:1428] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.047s user 0.008s sys 0.000s
I20260808 13:35:22.032217 6026 tablet_bootstrap.cc:492] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:22.037637 6026 tablet_bootstrap.cc:654] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:22.039424 6901 raft_consensus.cc:482] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:22.039929 6901 raft_consensus.cc:504] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.041886 6901 leader_election.cc:288] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:22.043124 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0c76b819474468baac51365360becfc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:22.044623 5551 leader_election.cc:341] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:21.953150 4835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_f212f0d1d7294b4f80606ba37c7f19f3" } modify_external_catalogs: true
I20260808 13:35:22.053246 4835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_f212f0d1d7294b4f80606ba37c7f19f3" } modify_external_catalogs: true
I20260808 13:35:22.026034 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet e7616d81aa924933a9b24d44004725fa (table default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9]) successfully deleted
I20260808 13:35:22.043466 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet d2ac4c366363468aa62b73e0f1e04532 (table default.Table_d81d55e146d84d41a68894ad0dd4dc93 [id=65666ad40b274ca9a277269157ccbb97]) successfully deleted
I20260808 13:35:22.058153 4837 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 239) took 1732 ms. Trace:
I20260808 13:35:22.058463 4837 rpcz_store.cc:276] 0808 13:35:20.325464 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.325590 (+ 126us) service_pool.cc:227] Handling call
0808 13:35:20.399904 (+ 74314us) spinlock_profiling.cc:243] Waited 72.9 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff7d6 00007f227bd45634 00007f227bd455c2 00007f227bd454f1 00007f227bd4546c 00007f227bd44722 00007f2279fde94c 00007f2279fddd86
0808 13:35:20.400573 (+ 669us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:20.401605 (+ 1032us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:20.402142 (+ 537us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:20.402315 (+ 173us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:20.837358 (+435043us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:22.058094 (+1220736us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:20.823758 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:20.824118 (+ 360us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:20.824145 (+ 27us) write_op.cc:435] Acquired schema lock
0808 13:35:20.824156 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:20.824404 (+ 248us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:20.824443 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:20.824473 (+ 30us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:20.824621 (+ 148us) tablet.cc:699] Row locks acquired
0808 13:35:20.824633 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:20.824735 (+ 102us) write_op.cc:270] Start()
0808 13:35:20.824828 (+ 93us) write_op.cc:276] Timestamp: P: 1786196120824634 usec, L: 0
0808 13:35:20.824842 (+ 14us) op_driver.cc:331] REPLICATION: starting
0808 13:35:20.825384 (+ 542us) log.cc:842] Serialized 405 byte log entry
0808 13:35:20.835642 (+ 10258us) op_driver.cc:444] REPLICATION: finished
0808 13:35:20.836107 (+ 465us) write_op.cc:301] APPLY: starting
0808 13:35:20.836167 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:20.836268 (+ 101us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:20.836379 (+ 111us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:20.836558 (+ 179us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:20.836696 (+ 138us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:20.836749 (+ 53us) write_op.cc:312] APPLY: finished
0808 13:35:20.836932 (+ 183us) log.cc:842] Serialized 23 byte log entry
0808 13:35:20.837068 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:20.837130 (+ 62us) write_op.cc:454] Released schema lock
0808 13:35:20.837161 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:20.837624 (+ 463us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:20.837649 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":167751552,"child_traces":[["op",{"apply.queue_time_us":238,"apply.run_cpu_time_us":1445,"apply.run_wall_time_us":1716,"num_ops":1,"prepare.queue_time_us":414532,"prepare.run_cpu_time_us":1966,"prepare.run_wall_time_us":44694,"raft.queue_time_us":102,"raft.run_cpu_time_us":1247,"raft.run_wall_time_us":1246,"replication_time_us":9737}]]}
I20260808 13:35:22.072479 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_ceeec8b3cee9414bad671191ab524f9d" } new_table_name: "default.Table_c4e53d8c393449429ef2791497ee7208" modify_external_catalogs: true
I20260808 13:35:22.081409 6833 raft_consensus.cc:482] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:22.081879 6833 raft_consensus.cc:504] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.083813 6833 leader_election.cc:288] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:22.026208 6587 rpcz_store.cc:276] 0808 13:35:20.116100 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.117136 (+ 1036us) service_pool.cc:227] Handling call
0808 13:35:22.025538 (+1908402us) ts_tablet_manager.cc:1145] Deleting tablet 2fa77744bcfb4a5cace730fd9f9fd1ca
0808 13:35:22.025589 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.025781 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1907622}
I20260808 13:35:22.089726 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 167) took 1973 ms. Trace:
I20260808 13:35:22.090023 6587 rpcz_store.cc:276] 0808 13:35:20.116394 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.118021 (+ 1627us) service_pool.cc:227] Handling call
0808 13:35:22.089469 (+1971448us) ts_tablet_manager.cc:1145] Deleting tablet e2a95712de254c13b3b6dc4fe5cb15f7
0808 13:35:22.089513 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.089672 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1970937}
I20260808 13:35:22.090991 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 168) took 1974 ms. Trace:
I20260808 13:35:22.091264 6587 rpcz_store.cc:276] 0808 13:35:20.116642 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.118592 (+ 1950us) service_pool.cc:227] Handling call
0808 13:35:22.090770 (+1972178us) ts_tablet_manager.cc:1145] Deleting tablet cfc9a6ec9a86431fb46e50fcaa658d33
0808 13:35:22.090804 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.090945 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1971340}
I20260808 13:35:22.092259 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 171) took 1706 ms. Trace:
I20260808 13:35:22.092535 6587 rpcz_store.cc:276] 0808 13:35:20.385663 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.390141 (+ 4478us) service_pool.cc:227] Handling call
0808 13:35:22.091952 (+1701811us) ts_tablet_manager.cc:1145] Deleting tablet 851a0b374a854efd87dd66e39b9df2d5
0808 13:35:22.091995 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.092216 (+ 221us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1700734}
I20260808 13:35:22.093545 6587 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 173) took 1158 ms. Trace:
I20260808 13:35:22.093823 6587 rpcz_store.cc:276] 0808 13:35:20.934875 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.935020 (+ 145us) service_pool.cc:227] Handling call
0808 13:35:22.093265 (+1158245us) ts_tablet_manager.cc:1145] Deleting tablet 69642fdba1f145438b04eae9f8154101
0808 13:35:22.093300 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.093504 (+ 204us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1157273}
I20260808 13:35:22.107568 6833 raft_consensus.cc:482] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:22.108050 6833 raft_consensus.cc:504] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.110697 6833 leader_election.cc:288] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:22.121397 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca because the tablet was not found. No further retry: Not found: Tablet not found: 2fa77744bcfb4a5cace730fd9f9fd1ca
W20260808 13:35:22.122956 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet e2a95712de254c13b3b6dc4fe5cb15f7 because the tablet was not found. No further retry: Not found: Tablet not found: e2a95712de254c13b3b6dc4fe5cb15f7
W20260808 13:35:22.125569 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet cfc9a6ec9a86431fb46e50fcaa658d33 because the tablet was not found. No further retry: Not found: Tablet not found: cfc9a6ec9a86431fb46e50fcaa658d33
W20260808 13:35:22.126856 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 851a0b374a854efd87dd66e39b9df2d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 851a0b374a854efd87dd66e39b9df2d5 already in progress: creating tablet
I20260808 13:35:22.133128 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e18e51c2e24d3caa3d29e6645e46ae" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:22.135504 5551 leader_election.cc:341] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:22.140362 6587 tablet_replica.cc:330] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:22.141295 6587 raft_consensus.cc:2229] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:22.141917 6587 raft_consensus.cc:2258] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:22.144461 6737 raft_consensus.cc:482] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:22.145077 6737 raft_consensus.cc:504] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.154469 6587 ts_tablet_manager.cc:1905] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:22.180929 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 152) took 4115 ms. Trace:
I20260808 13:35:22.181397 6585 rpcz_store.cc:276] 0808 13:35:18.065443 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.065634 (+ 191us) service_pool.cc:227] Handling call
0808 13:35:22.155398 (+4089764us) ts_tablet_manager.cc:1145] Deleting tablet 78d492406c8a4c4186131e02a1559cb4
0808 13:35:22.180571 (+ 25173us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.180855 (+ 284us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4064667}
I20260808 13:35:22.184543 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 153) took 4117 ms. Trace:
I20260808 13:35:22.185134 6585 rpcz_store.cc:276] 0808 13:35:18.067272 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.067405 (+ 133us) service_pool.cc:227] Handling call
0808 13:35:22.184166 (+4116761us) ts_tablet_manager.cc:1145] Deleting tablet edfa66acceea4b54911d78ee16609ef7
0808 13:35:22.184231 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.184475 (+ 244us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4116217}
I20260808 13:35:22.213421 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "973a7afdcbf944aab8f11b3fd3995865" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:22.217813 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "973a7afdcbf944aab8f11b3fd3995865" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:22.219951 6587 ts_tablet_manager.cc:1918] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:22.220309 6587 log.cc:1196] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/60f8863bc32e49399940912444bcb972
I20260808 13:35:22.221006 6587 ts_tablet_manager.cc:1939] T 60f8863bc32e49399940912444bcb972 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:22.237844 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 158) took 3623 ms. Trace:
I20260808 13:35:22.238416 6585 rpcz_store.cc:276] 0808 13:35:18.614358 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.614512 (+ 154us) service_pool.cc:227] Handling call
0808 13:35:22.237284 (+3622772us) ts_tablet_manager.cc:1145] Deleting tablet 8b57966a9b7b4673ab9f98720cfd456d
0808 13:35:22.237352 (+ 68us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.237771 (+ 419us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3614754}
I20260808 13:35:22.262928 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9260370cf8cb46a7b699581e4e8d46db" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:22.264997 5551 leader_election.cc:341] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:22.219204 6033 tablet_bootstrap.cc:492] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:22.268662 6033 ts_tablet_manager.cc:1397] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.919s user 0.014s sys 0.007s
I20260808 13:35:22.210917 6737 leader_election.cc:288] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:22.220438 5772 leader_election.cc:302] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:22.274778 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 54609.4us
I20260808 13:35:22.275641 6737 raft_consensus.cc:2726] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:22.278446 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 157) took 3682 ms. Trace:
I20260808 13:35:22.279053 6585 rpcz_store.cc:276] 0808 13:35:18.596155 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:18.596350 (+ 195us) service_pool.cc:227] Handling call
0808 13:35:22.277912 (+3681562us) ts_tablet_manager.cc:1145] Deleting tablet 264ee45f8dfa4a05b0e5e812ce1b622c
0808 13:35:22.277986 (+ 74us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.278380 (+ 394us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3646028}
I20260808 13:35:22.281214 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 164) took 3216 ms. Trace:
W20260808 13:35:22.183422 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 78d492406c8a4c4186131e02a1559cb4 because the tablet was not found. No further retry: Not found: Tablet not found: 78d492406c8a4c4186131e02a1559cb4
W20260808 13:35:22.282397 4819 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 99015.2us
W20260808 13:35:22.284641 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet edfa66acceea4b54911d78ee16609ef7 because the tablet was not found. No further retry: Not found: Tablet not found: edfa66acceea4b54911d78ee16609ef7
W20260808 13:35:22.285363 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 8b57966a9b7b4673ab9f98720cfd456d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b57966a9b7b4673ab9f98720cfd456d already in progress: creating tablet
W20260808 13:35:22.286036 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 264ee45f8dfa4a05b0e5e812ce1b622c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 264ee45f8dfa4a05b0e5e812ce1b622c already in progress: creating tablet
W20260808 13:35:22.287058 4820 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 160199us
W20260808 13:35:22.288102 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 69642fdba1f145438b04eae9f8154101 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69642fdba1f145438b04eae9f8154101 already in progress: creating tablet
W20260808 13:35:22.288950 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 1f9bc338093f403881d329a7e913c9f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f9bc338093f403881d329a7e913c9f7 already in progress: creating tablet
W20260808 13:35:22.290338 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 369a942248bd4cec9ee9170fe55601a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 369a942248bd4cec9ee9170fe55601a3 already in progress: creating tablet
I20260808 13:35:22.301223 4842 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_31cc5d89c86849ae876aa34027918bbc [id=d5b4528cac46486e857db6176b4b3a2b] alter complete (version 2)
W20260808 13:35:22.260768 4837 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4b7284afe0294644a368cf51451c7ab6 consensus state has no leader
I20260808 13:35:22.372565 6026 tablet_bootstrap.cc:492] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:22.373239 6026 ts_tablet_manager.cc:1397] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.341s user 0.019s sys 0.040s
I20260808 13:35:22.376713 6026 raft_consensus.cc:348] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.377698 6026 raft_consensus.cc:374] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:22.378029 6026 raft_consensus.cc:729] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
W20260808 13:35:22.378939 5550 consensus_peers.cc:487] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:22.378770 6026 consensus_queue.cc:260] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.385842 6026 ts_tablet_manager.cc:1428] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.012s user 0.006s sys 0.003s
I20260808 13:35:22.386693 6585 rpcz_store.cc:276] 0808 13:35:19.064581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.065757 (+ 1176us) service_pool.cc:227] Handling call
0808 13:35:22.280647 (+3214890us) ts_tablet_manager.cc:1145] Deleting tablet 3a27517560d64a3b8da2cf55980e18a8
0808 13:35:22.280711 (+ 64us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.281151 (+ 440us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3214236}
I20260808 13:35:22.391710 6585 tablet_replica.cc:330] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:22.392465 6585 raft_consensus.cc:2229] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:22.393354 6585 raft_consensus.cc:2258] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:22.395677 6026 tablet_bootstrap.cc:492] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:22.412007 5550 leader_election.cc:341] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:22.412492 5550 leader_election.cc:302] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:22.413309 6901 raft_consensus.cc:2726] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:22.416123 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0c76b819474468baac51365360becfc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:22.418720 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9260370cf8cb46a7b699581e4e8d46db" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:22.418344 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e18e51c2e24d3caa3d29e6645e46ae" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:22.431926 6796 raft_consensus.cc:482] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:22.433130 6796 raft_consensus.cc:504] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.437631 6796 leader_election.cc:288] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:22.438867 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd2d0c9ab2c4704a74d06559bb913d6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:22.439504 5637 raft_consensus.cc:2454] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:22.295506 6033 raft_consensus.cc:348] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.445960 6033 raft_consensus.cc:374] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:22.446293 6033 raft_consensus.cc:729] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:22.454735 6585 ts_tablet_manager.cc:1905] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:22.457667 6033 consensus_queue.cc:260] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.466964 6033 ts_tablet_manager.cc:1428] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.198s user 0.006s sys 0.000s
W20260808 13:35:22.466919 5550 leader_election.cc:341] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:22.467372 5550 leader_election.cc:302] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:22.468513 5550 leader_election.cc:341] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:22.468878 5550 leader_election.cc:302] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:22.477265 6937 raft_consensus.cc:2726] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:22.484033 5285 leader_election.cc:302] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:22.485276 6913 raft_consensus.cc:2781] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:22.485723 6913 raft_consensus.cc:482] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:22.486114 6913 raft_consensus.cc:3037] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:22.493716 6046 tablet_bootstrap.cc:492] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
W20260808 13:35:22.382333 5833 consensus_meta.cc:287] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.542s user 0.002s sys 0.003s
I20260808 13:35:22.483588 6833 raft_consensus.cc:2726] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:22.496487 6937 consensus_queue.cc:1035] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260808 13:35:22.481590 6026 tablet_bootstrap.cc:654] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:22.495764 5550 leader_election.cc:341] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:22.335597 5845 consensus_meta.cc:287] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.718s user 0.011s sys 0.000s
I20260808 13:35:22.507012 5550 leader_election.cc:302] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:22.516086 6046 ts_tablet_manager.cc:1397] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.613s user 0.113s sys 0.162s
W20260808 13:35:22.520265 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_c4e53d8c393449429ef2791497ee7208 [id=26fb789fff434bb0825daf68d5378013], current schema version=5) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4b7284afe0294644a368cf51451c7ab6 consensus state has no leader
I20260808 13:35:22.520948 6947 tablet.cc:1722] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260808 13:35:22.521252 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 3a27517560d64a3b8da2cf55980e18a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a27517560d64a3b8da2cf55980e18a8 already in progress: creating tablet
W20260808 13:35:22.524258 4820 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 233817us
I20260808 13:35:22.506120 6938 tablet.cc:1722] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:22.525087 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 60f8863bc32e49399940912444bcb972 (table default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]) successfully deleted
I20260808 13:35:22.534261 6937 raft_consensus.cc:2726] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:22.534828 6937 raft_consensus.cc:482] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:22.535183 6937 raft_consensus.cc:504] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.537124 6937 leader_election.cc:288] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:22.559304 6585 ts_tablet_manager.cc:1918] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:22.559790 6585 log.cc:1196] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/6d8018bf8ff843d8818fbba9473acf47
I20260808 13:35:22.560528 6585 ts_tablet_manager.cc:1939] T 6d8018bf8ff843d8818fbba9473acf47 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:22.591843 6046 raft_consensus.cc:348] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.592600 6046 raft_consensus.cc:374] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:22.592914 6046 raft_consensus.cc:729] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:22.593470 6046 consensus_queue.cc:260] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.597174 6946 tablet.cc:1722] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:22.606946 5845 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 174) took 1045 ms. Trace:
I20260808 13:35:22.607079 6033 tablet_bootstrap.cc:492] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:22.607263 5845 rpcz_store.cc:276] 0808 13:35:21.563739 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:21.592393 (+ 28654us) service_pool.cc:227] Handling call
0808 13:35:21.593891 (+ 1498us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:21.593976 (+ 85us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:21.616525 (+ 22549us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:22.603040 (+986515us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:22.606888 (+ 3848us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3349}
I20260808 13:35:22.614475 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1947f03e107447bb8e23fe5deaf1ea60" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:22.621776 5550 leader_election.cc:341] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:22.630121 6947 tablet.cc:1722] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:22.633057 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1947f03e107447bb8e23fe5deaf1ea60" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:22.634425 6046 ts_tablet_manager.cc:1428] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.118s user 0.006s sys 0.001s
I20260808 13:35:22.614276 6913 raft_consensus.cc:504] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.643965 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd2d0c9ab2c4704a74d06559bb913d6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:22.644534 5642 raft_consensus.cc:3037] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:22.647017 6913 leader_election.cc:288] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:22.650938 5551 leader_election.cc:341] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:22.651461 5551 leader_election.cc:302] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:22.652311 6937 raft_consensus.cc:2726] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:22.656421 6046 tablet_bootstrap.cc:492] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:22.659469 6033 tablet_bootstrap.cc:654] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:22.670733 5642 raft_consensus.cc:2454] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:22.684080 5285 leader_election.cc:302] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:22.685479 6913 raft_consensus.cc:2781] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:22.689314 6913 raft_consensus.cc:686] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:22.690184 6913 consensus_queue.cc:237] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.702801 6901 consensus_queue.cc:1035] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:22.706382 6951 tablet.cc:1722] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:22.724323 6026 tablet_bootstrap.cc:492] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:22.724728 6026 ts_tablet_manager.cc:1397] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.329s user 0.037s sys 0.043s
I20260808 13:35:22.727139 6026 raft_consensus.cc:348] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.727767 6026 raft_consensus.cc:374] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:22.728015 6026 raft_consensus.cc:729] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:22.728762 6026 consensus_queue.cc:260] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.730703 6026 ts_tablet_manager.cc:1428] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 13:35:22.733502 6937 raft_consensus.cc:482] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:22.733923 6937 raft_consensus.cc:504] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.735858 6937 leader_election.cc:288] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:22.739550 6026 tablet_bootstrap.cc:492] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:22.740127 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c083c8e19d4279af600670f5d1d78d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:22.741515 5551 leader_election.cc:341] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:22.745398 6026 tablet_bootstrap.cc:654] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:22.666903 6737 raft_consensus.cc:482] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:22.754825 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 165) took 3689 ms. Trace:
I20260808 13:35:22.755142 6585 rpcz_store.cc:276] 0808 13:35:19.064963 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.066508 (+ 1545us) service_pool.cc:227] Handling call
0808 13:35:22.391526 (+3325018us) ts_tablet_manager.cc:1145] Deleting tablet 6d8018bf8ff843d8818fbba9473acf47
0808 13:35:22.391588 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.518805 (+127217us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:22.559053 (+ 40248us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:22.754766 (+195713us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3308242}
I20260808 13:35:22.756436 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 167) took 3159 ms. Trace:
I20260808 13:35:22.756727 6585 rpcz_store.cc:276] 0808 13:35:19.597129 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.597397 (+ 268us) service_pool.cc:227] Handling call
0808 13:35:22.756029 (+3158632us) ts_tablet_manager.cc:1145] Deleting tablet e992a0294c404cfb9541b57782a05fc9
0808 13:35:22.756082 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.756386 (+ 304us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3157806}
I20260808 13:35:22.757817 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 169) took 2963 ms. Trace:
I20260808 13:35:22.761737 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 6d8018bf8ff843d8818fbba9473acf47 (table default.Table_810abec89b21407191098eb04ea650e2 [id=fc56814206154b1c9d77c3b160d25aa8]) successfully deleted
I20260808 13:35:22.758090 6585 rpcz_store.cc:276] 0808 13:35:19.794672 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.798734 (+ 4062us) service_pool.cc:227] Handling call
0808 13:35:22.757469 (+2958735us) ts_tablet_manager.cc:1145] Deleting tablet d3682c9c656c4556bc13ae3aeddd7c5e
0808 13:35:22.757517 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.757772 (+ 255us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2958106}
W20260808 13:35:22.769381 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet e992a0294c404cfb9541b57782a05fc9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e992a0294c404cfb9541b57782a05fc9 already in progress: creating tablet
W20260808 13:35:22.770215 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet d3682c9c656c4556bc13ae3aeddd7c5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3682c9c656c4556bc13ae3aeddd7c5e already in progress: creating tablet
I20260808 13:35:22.771333 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 168) took 3008 ms. Trace:
I20260808 13:35:22.752929 6737 raft_consensus.cc:504] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.771718 6585 rpcz_store.cc:276] 0808 13:35:19.763418 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.809292 (+ 45874us) service_pool.cc:227] Handling call
0808 13:35:22.770883 (+2961591us) ts_tablet_manager.cc:1145] Deleting tablet 9423b380a62c467895e7fdff14756586
0808 13:35:22.770939 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.771277 (+ 338us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2953413}
I20260808 13:35:22.773146 6585 tablet_replica.cc:330] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
W20260808 13:35:22.773693 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 9423b380a62c467895e7fdff14756586 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9423b380a62c467895e7fdff14756586 already in progress: creating tablet
I20260808 13:35:22.773604 6737 leader_election.cc:288] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:22.787328 6585 raft_consensus.cc:2229] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:22.788697 6585 raft_consensus.cc:2258] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:22.790537 6585 ts_tablet_manager.cc:1905] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:22.795583 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4625b632d67244fc86e48d739dde707e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:22.795868 6033 tablet_bootstrap.cc:492] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:22.796294 6033 ts_tablet_manager.cc:1397] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.189s user 0.007s sys 0.008s
I20260808 13:35:22.800385 6046 tablet_bootstrap.cc:654] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:22.800261 6033 raft_consensus.cc:348] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.800896 6033 raft_consensus.cc:374] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:22.801174 6033 raft_consensus.cc:729] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:22.804176 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4625b632d67244fc86e48d739dde707e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:22.805130 6033 consensus_queue.cc:260] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.808372 6033 ts_tablet_manager.cc:1428] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
I20260808 13:35:22.809300 6033 tablet_bootstrap.cc:492] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:22.814359 4823 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:37135, user_credentials={real_user=slave}} blocked reactor thread for 445698us
I20260808 13:35:22.827150 6952 tablet.cc:1722] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260808 13:35:22.835090 6964 log.cc:924] Time spent T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c: Append to log took a long time: real 0.073s user 0.001s sys 0.000s
W20260808 13:35:22.836973 5283 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.907s user 0.003s sys 0.013s
W20260808 13:35:22.837407 5283 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.908s user 0.003s sys 0.013s
W20260808 13:35:22.838059 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 909213us
I20260808 13:35:22.841125 5882 raft_consensus.cc:3037] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:22.836534 5772 leader_election.cc:302] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:22.846813 6033 tablet_bootstrap.cc:654] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:22.861996 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c083c8e19d4279af600670f5d1d78d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:22.879055 6946 tablet.cc:1722] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260808 13:35:22.879388 5550 leader_election.cc:341] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:22.879840 5550 leader_election.cc:302] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:22.881033 6944 raft_consensus.cc:2726] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:22.882843 6046 tablet_bootstrap.cc:492] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:22.883338 6046 ts_tablet_manager.cc:1397] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.227s user 0.012s sys 0.000s
I20260808 13:35:22.884016 6585 ts_tablet_manager.cc:1918] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:22.884367 6585 log.cc:1196] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/e7616d81aa924933a9b24d44004725fa
I20260808 13:35:22.885010 6585 ts_tablet_manager.cc:1939] T e7616d81aa924933a9b24d44004725fa P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:22.885942 6046 raft_consensus.cc:348] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.897439 6046 raft_consensus.cc:374] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:22.897744 6046 raft_consensus.cc:729] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:22.898425 6046 consensus_queue.cc:260] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.900985 6046 ts_tablet_manager.cc:1428] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.017s user 0.004s sys 0.002s
I20260808 13:35:22.901870 6046 tablet_bootstrap.cc:492] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:22.907944 6033 tablet_bootstrap.cc:492] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:22.908366 6033 ts_tablet_manager.cc:1397] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.099s user 0.004s sys 0.004s
I20260808 13:35:22.910248 5878 raft_consensus.cc:3037] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:22.910369 6033 raft_consensus.cc:348] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.912387 6033 raft_consensus.cc:374] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:22.912670 6033 raft_consensus.cc:729] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:22.913456 6033 consensus_queue.cc:260] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:22.929631 6033 ts_tablet_manager.cc:1428] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.021s user 0.007s sys 0.001s
I20260808 13:35:22.935307 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd2d0c9ab2c4704a74d06559bb913d6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:22.936173 5879 raft_consensus.cc:2454] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:22.941174 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 170) took 3001 ms. Trace:
I20260808 13:35:22.941504 6585 rpcz_store.cc:276] 0808 13:35:19.940103 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.945082 (+ 4979us) service_pool.cc:227] Handling call
0808 13:35:22.772992 (+2827910us) ts_tablet_manager.cc:1145] Deleting tablet e7616d81aa924933a9b24d44004725fa
0808 13:35:22.773043 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.837409 (+ 64366us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:22.883794 (+ 46385us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:22.941113 (+ 57319us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2826858}
I20260808 13:35:22.944156 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet e7616d81aa924933a9b24d44004725fa (table default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9]) successfully deleted
I20260808 13:35:22.947224 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 171) took 2491 ms. Trace:
I20260808 13:35:22.947511 6585 rpcz_store.cc:276] 0808 13:35:20.456336 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.456484 (+ 148us) service_pool.cc:227] Handling call
0808 13:35:22.946844 (+2490360us) ts_tablet_manager.cc:1145] Deleting tablet 851a0b374a854efd87dd66e39b9df2d5
0808 13:35:22.946905 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.947173 (+ 268us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2489679}
W20260808 13:35:22.948567 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 851a0b374a854efd87dd66e39b9df2d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 851a0b374a854efd87dd66e39b9df2d5 already in progress: creating tablet
I20260808 13:35:22.948947 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 172) took 2017 ms. Trace:
I20260808 13:35:22.949229 6585 rpcz_store.cc:276] 0808 13:35:20.931823 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.932069 (+ 246us) service_pool.cc:227] Handling call
0808 13:35:22.948568 (+2016499us) ts_tablet_manager.cc:1145] Deleting tablet 69642fdba1f145438b04eae9f8154101
0808 13:35:22.948616 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.948890 (+ 274us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2015775}
W20260808 13:35:22.950568 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 69642fdba1f145438b04eae9f8154101 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69642fdba1f145438b04eae9f8154101 already in progress: creating tablet
I20260808 13:35:22.953436 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 177) took 1381 ms. Trace:
I20260808 13:35:22.953712 6585 rpcz_store.cc:276] 0808 13:35:21.572254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:21.590945 (+ 18691us) service_pool.cc:227] Handling call
0808 13:35:22.953139 (+1362194us) ts_tablet_manager.cc:1145] Deleting tablet 1f9bc338093f403881d329a7e913c9f7
0808 13:35:22.953175 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.953389 (+ 214us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1361288}
I20260808 13:35:22.954105 6046 tablet_bootstrap.cc:654] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:22.954711 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 1f9bc338093f403881d329a7e913c9f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f9bc338093f403881d329a7e913c9f7 already in progress: creating tablet
I20260808 13:35:22.955853 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd2d0c9ab2c4704a74d06559bb913d6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:22.956655 5879 raft_consensus.cc:3037] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:22.957819 6033 tablet_bootstrap.cc:492] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:22.959117 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 178) took 1375 ms. Trace:
I20260808 13:35:22.959412 6585 rpcz_store.cc:276] 0808 13:35:21.584098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:21.590913 (+ 6815us) service_pool.cc:227] Handling call
0808 13:35:22.958808 (+1367895us) ts_tablet_manager.cc:1145] Deleting tablet 369a942248bd4cec9ee9170fe55601a3
0808 13:35:22.958850 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:22.959071 (+ 221us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1366630}
W20260808 13:35:22.961591 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 369a942248bd4cec9ee9170fe55601a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 369a942248bd4cec9ee9170fe55601a3 already in progress: creating tablet
I20260808 13:35:22.961740 6585 tablet_replica.cc:330] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:22.963362 6585 raft_consensus.cc:2229] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:22.963893 6585 raft_consensus.cc:2258] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:22.978284 6585 ts_tablet_manager.cc:1905] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:22.983773 6026 tablet_bootstrap.cc:492] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:22.984179 6026 ts_tablet_manager.cc:1397] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.245s user 0.029s sys 0.017s
I20260808 13:35:22.986438 6026 raft_consensus.cc:348] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:22.991364 6026 raft_consensus.cc:374] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:22.991708 6026 raft_consensus.cc:729] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:22.992313 6033 tablet_bootstrap.cc:654] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:22.994843 6026 consensus_queue.cc:260] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.001343 6951 tablet.cc:1722] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:23.003837 6026 ts_tablet_manager.cc:1428] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260808 13:35:23.004662 6026 tablet_bootstrap.cc:492] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:23.008819 5879 raft_consensus.cc:2454] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:23.018177 6046 tablet_bootstrap.cc:492] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:23.018775 6046 ts_tablet_manager.cc:1397] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.117s user 0.011s sys 0.004s
I20260808 13:35:23.021229 6913 raft_consensus.cc:482] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.021126 6046 raft_consensus.cc:348] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.021962 6046 raft_consensus.cc:374] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.021801 6913 raft_consensus.cc:504] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:23.022254 6046 raft_consensus.cc:729] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.023211 6046 consensus_queue.cc:260] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.023813 6913 leader_election.cc:288] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:23.033890 5285 leader_election.cc:341] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: a0b29786cb404bc2b01473c3b8390a50
W20260808 13:35:23.039762 5283 leader_election.cc:341] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: a0b29786cb404bc2b01473c3b8390a50
I20260808 13:35:23.040158 5283 leader_election.cc:302] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:23.040999 6793 raft_consensus.cc:2726] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:23.041548 6913 consensus_queue.cc:1035] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:23.043998 6026 tablet_bootstrap.cc:654] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.050752 6810 raft_consensus.cc:2726] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:23.057703 6585 ts_tablet_manager.cc:1918] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:23.058070 6585 log.cc:1196] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/60f8863bc32e49399940912444bcb972
I20260808 13:35:23.069799 6046 ts_tablet_manager.cc:1428] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.051s user 0.004s sys 0.002s
I20260808 13:35:23.072445 6033 tablet_bootstrap.cc:492] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:23.072885 6033 ts_tablet_manager.cc:1397] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.115s user 0.011s sys 0.000s
I20260808 13:35:23.076514 6026 tablet_bootstrap.cc:492] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:23.077339 6981 tablet.cc:1722] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:23.076946 6026 ts_tablet_manager.cc:1397] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.072s user 0.007s sys 0.002s
I20260808 13:35:23.079056 6033 raft_consensus.cc:348] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:23.079752 6033 raft_consensus.cc:374] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.080001 6033 raft_consensus.cc:729] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.080991 6026 raft_consensus.cc:348] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.081604 6026 raft_consensus.cc:374] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.081787 6046 tablet_bootstrap.cc:492] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:23.081847 6026 raft_consensus.cc:729] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.082522 6585 ts_tablet_manager.cc:1939] T 60f8863bc32e49399940912444bcb972 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:23.083210 6026 consensus_queue.cc:260] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.100414 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 186) took 1199 ms. Trace:
I20260808 13:35:23.100755 6585 rpcz_store.cc:276] 0808 13:35:21.901260 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:21.914794 (+ 13534us) service_pool.cc:227] Handling call
0808 13:35:22.961606 (+1046812us) ts_tablet_manager.cc:1145] Deleting tablet 60f8863bc32e49399940912444bcb972
0808 13:35:22.961640 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:23.027761 (+ 66121us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:23.057488 (+ 29727us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:23.100348 (+ 42860us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1046205}
I20260808 13:35:23.102051 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 60f8863bc32e49399940912444bcb972 (table default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]) successfully deleted
I20260808 13:35:23.107306 6026 ts_tablet_manager.cc:1428] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.028s user 0.003s sys 0.002s
I20260808 13:35:23.108270 6026 tablet_bootstrap.cc:492] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:23.115060 6046 tablet_bootstrap.cc:654] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.080587 6033 consensus_queue.cc:260] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:23.118893 6913 raft_consensus.cc:482] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.119339 6913 raft_consensus.cc:504] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.121189 6913 leader_election.cc:288] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:23.122905 6026 tablet_bootstrap.cc:654] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.123602 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1947f03e107447bb8e23fe5deaf1ea60" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:23.123908 6033 ts_tablet_manager.cc:1428] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.051s user 0.005s sys 0.000s
W20260808 13:35:23.124827 5283 leader_election.cc:341] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.126889 6033 tablet_bootstrap.cc:492] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:23.123736 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1947f03e107447bb8e23fe5deaf1ea60" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:23.129395 5637 raft_consensus.cc:2454] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:23.130859 5285 leader_election.cc:302] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:23.132148 6913 raft_consensus.cc:2781] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:23.132498 6913 raft_consensus.cc:482] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:23.132843 6913 raft_consensus.cc:3037] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.139994 6913 raft_consensus.cc:504] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.151932 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1947f03e107447bb8e23fe5deaf1ea60" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:23.153388 5283 leader_election.cc:341] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.154150 6033 tablet_bootstrap.cc:654] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.154775 6046 tablet_bootstrap.cc:492] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:23.155081 6913 leader_election.cc:288] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:23.156236 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1947f03e107447bb8e23fe5deaf1ea60" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:23.155182 6046 ts_tablet_manager.cc:1397] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.074s user 0.010s sys 0.000s
I20260808 13:35:23.156852 5637 raft_consensus.cc:3037] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.159379 6046 raft_consensus.cc:348] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.160177 6046 raft_consensus.cc:374] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.160540 6046 raft_consensus.cc:729] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.165544 5637 raft_consensus.cc:2454] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:23.166648 5285 leader_election.cc:302] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:23.170364 6913 raft_consensus.cc:2781] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:23.171617 6913 raft_consensus.cc:686] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:23.175361 6913 consensus_queue.cc:237] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.178897 6046 consensus_queue.cc:260] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.184028 6046 ts_tablet_manager.cc:1428] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260808 13:35:23.204335 6026 tablet_bootstrap.cc:492] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:23.204797 6026 ts_tablet_manager.cc:1397] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.097s user 0.009s sys 0.005s
I20260808 13:35:23.206131 6046 tablet_bootstrap.cc:492] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:23.206993 6026 raft_consensus.cc:348] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.207585 6026 raft_consensus.cc:374] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.207829 6026 raft_consensus.cc:729] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.208468 6026 consensus_queue.cc:260] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.212258 6026 ts_tablet_manager.cc:1428] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260808 13:35:23.213645 6033 tablet_bootstrap.cc:492] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:23.214036 6033 ts_tablet_manager.cc:1397] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.087s user 0.019s sys 0.012s
I20260808 13:35:23.216714 6033 raft_consensus.cc:348] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.217335 6033 raft_consensus.cc:374] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.217574 6033 raft_consensus.cc:729] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.218345 6033 consensus_queue.cc:260] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.213435 6026 tablet_bootstrap.cc:492] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:23.223094 6033 ts_tablet_manager.cc:1428] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.009s user 0.002s sys 0.003s
I20260808 13:35:23.226898 6033 tablet_bootstrap.cc:492] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:23.233201 6026 tablet_bootstrap.cc:654] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.233367 6033 tablet_bootstrap.cc:654] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.238514 6046 tablet_bootstrap.cc:654] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.250698 6913 raft_consensus.cc:482] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.250929 6937 raft_consensus.cc:482] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.251250 6913 raft_consensus.cc:504] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:23.251389 6937 raft_consensus.cc:504] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.254264 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a26598a69146e1950e9227523fff13" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:23.254948 5637 raft_consensus.cc:2454] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:23.255661 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f1618a1d044898bffbc82107e90840" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:23.255694 6033 tablet_bootstrap.cc:492] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:23.256157 5285 leader_election.cc:302] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:23.256812 6033 ts_tablet_manager.cc:1397] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.002s
I20260808 13:35:23.257014 6793 raft_consensus.cc:2781] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:23.257364 6793 raft_consensus.cc:482] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:23.257686 6937 leader_election.cc:288] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:23.257812 6793 raft_consensus.cc:3037] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.258965 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f1618a1d044898bffbc82107e90840" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:23.260787 5550 leader_election.cc:341] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.261648 6033 raft_consensus.cc:348] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:23.262167 5551 leader_election.cc:341] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.262359 6033 raft_consensus.cc:374] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.262517 5551 leader_election.cc:302] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:23.263566 6937 raft_consensus.cc:2726] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:23.264649 6033 raft_consensus.cc:729] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.265852 6033 consensus_queue.cc:260] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.267102 6793 raft_consensus.cc:504] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:23.268016 6033 ts_tablet_manager.cc:1428] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260808 13:35:23.270239 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a26598a69146e1950e9227523fff13" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:23.271055 5637 raft_consensus.cc:3037] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.275224 6026 tablet_bootstrap.cc:492] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:23.275362 6937 raft_consensus.cc:482] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.275651 6026 ts_tablet_manager.cc:1397] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.062s user 0.000s sys 0.009s
I20260808 13:35:23.272280 6033 tablet_bootstrap.cc:492] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:23.275837 6937 raft_consensus.cc:504] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.278172 6026 raft_consensus.cc:348] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.278820 6026 raft_consensus.cc:374] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.279058 6026 raft_consensus.cc:729] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.279003 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ea98afef6841fead641b314ee53a6e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:23.280361 5551 leader_election.cc:341] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.281533 6026 consensus_queue.cc:260] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.282323 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ea98afef6841fead641b314ee53a6e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:23.282043 6937 leader_election.cc:288] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:23.284281 5550 leader_election.cc:341] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.284406 6810 raft_consensus.cc:482] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.284812 6810 raft_consensus.cc:504] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:23.284799 5550 leader_election.cc:302] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:23.288903 6046 tablet_bootstrap.cc:492] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:23.289652 6046 ts_tablet_manager.cc:1397] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.084s user 0.006s sys 0.006s
I20260808 13:35:23.291603 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0c76b819474468baac51365360becfc" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:23.292114 6937 raft_consensus.cc:2726] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:23.292244 5386 raft_consensus.cc:2454] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:23.292865 6026 ts_tablet_manager.cc:1428] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260808 13:35:23.294185 5771 leader_election.cc:302] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:23.295301 6953 raft_consensus.cc:2781] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:23.295629 6953 raft_consensus.cc:482] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:23.295936 6953 raft_consensus.cc:3037] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.298902 6810 leader_election.cc:288] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:23.300319 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0c76b819474468baac51365360becfc" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:23.300884 5642 raft_consensus.cc:2454] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:23.303078 5637 raft_consensus.cc:2454] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:23.304224 5285 leader_election.cc:302] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:23.305044 6796 raft_consensus.cc:2781] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:23.305871 6913 leader_election.cc:288] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:23.306836 6937 raft_consensus.cc:482] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.306259 6953 raft_consensus.cc:504] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:23.307335 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a26598a69146e1950e9227523fff13" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:23.307281 6937 raft_consensus.cc:504] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.307928 5879 raft_consensus.cc:2454] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:23.309104 6937 leader_election.cc:288] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:23.309276 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0c76b819474468baac51365360becfc" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:23.309868 5386 raft_consensus.cc:3037] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.311072 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e7f2daaad4f48a79b94344ff35e98" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:23.312393 5551 leader_election.cc:341] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.312956 6793 leader_election.cc:288] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:23.318495 6953 leader_election.cc:288] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:23.319272 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a26598a69146e1950e9227523fff13" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:23.319823 5880 raft_consensus.cc:3037] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.320415 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0c76b819474468baac51365360becfc" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:23.320423 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e7f2daaad4f48a79b94344ff35e98" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:23.320986 5637 raft_consensus.cc:3037] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:23.321854 5550 leader_election.cc:341] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.322250 5550 leader_election.cc:302] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:23.322969 6937 raft_consensus.cc:2726] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:23.327529 6026 tablet_bootstrap.cc:492] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:23.329506 5880 raft_consensus.cc:2454] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:23.336174 6033 tablet_bootstrap.cc:654] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.339780 6026 tablet_bootstrap.cc:654] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.321563 6046 raft_consensus.cc:348] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.351445 6046 raft_consensus.cc:374] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.351799 6046 raft_consensus.cc:729] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.353374 6796 raft_consensus.cc:686] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:23.317502 5386 raft_consensus.cc:2454] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:23.355063 6796 consensus_queue.cc:237] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:23.355760 5771 leader_election.cc:302] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:23.356107 6033 tablet_bootstrap.cc:492] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:23.356750 6953 raft_consensus.cc:2781] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:23.357287 6953 raft_consensus.cc:686] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:23.358104 6953 consensus_queue.cc:237] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:23.350764 5637 raft_consensus.cc:2454] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:23.356959 6033 ts_tablet_manager.cc:1397] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.088s user 0.007s sys 0.010s
I20260808 13:35:23.361544 6046 consensus_queue.cc:260] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.365545 6046 ts_tablet_manager.cc:1428] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.075s user 0.005s sys 0.001s
I20260808 13:35:23.366439 6046 tablet_bootstrap.cc:492] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:23.365567 6033 raft_consensus.cc:348] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.367028 6033 raft_consensus.cc:374] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.367480 6033 raft_consensus.cc:729] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.368052 6999 raft_consensus.cc:482] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.368485 6999 raft_consensus.cc:504] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.368928 6033 consensus_queue.cc:260] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.371945 6999 leader_election.cc:288] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:23.374435 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9260370cf8cb46a7b699581e4e8d46db" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:23.374406 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9260370cf8cb46a7b699581e4e8d46db" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:23.375218 5637 raft_consensus.cc:2454] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
W20260808 13:35:23.376461 5283 leader_election.cc:341] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.377370 5285 leader_election.cc:302] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:23.379086 6999 raft_consensus.cc:2781] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:23.379756 6999 raft_consensus.cc:482] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:23.380312 6999 raft_consensus.cc:3037] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.389453 6033 ts_tablet_manager.cc:1428] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.027s user 0.006s sys 0.002s
I20260808 13:35:23.394423 6033 tablet_bootstrap.cc:492] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:23.400863 6046 tablet_bootstrap.cc:654] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.405488 6999 raft_consensus.cc:504] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.408723 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9260370cf8cb46a7b699581e4e8d46db" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:23.410365 5283 leader_election.cc:341] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.413653 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9260370cf8cb46a7b699581e4e8d46db" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:23.414638 5637 raft_consensus.cc:3037] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.429149 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 96bed5e6d1aa49a9b14ba86403c985ce (DEFAULT_TABLE table=default.Table_d3498dfbc8c04e42abefc8e337816f9b [id=1816314c549642e3adc9f39e532ab1ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.430491 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96bed5e6d1aa49a9b14ba86403c985ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.432439 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 96bed5e6d1aa49a9b14ba86403c985ce (DEFAULT_TABLE table=default.Table_d3498dfbc8c04e42abefc8e337816f9b [id=1816314c549642e3adc9f39e532ab1ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.432585 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 96bed5e6d1aa49a9b14ba86403c985ce (DEFAULT_TABLE table=default.Table_d3498dfbc8c04e42abefc8e337816f9b [id=1816314c549642e3adc9f39e532ab1ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.433499 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96bed5e6d1aa49a9b14ba86403c985ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.433514 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96bed5e6d1aa49a9b14ba86403c985ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.440443 6046 tablet_bootstrap.cc:492] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:23.440858 6046 ts_tablet_manager.cc:1397] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.075s user 0.006s sys 0.003s
I20260808 13:35:23.443490 6046 raft_consensus.cc:348] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.446002 6937 raft_consensus.cc:482] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.446671 6937 raft_consensus.cc:504] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.452411 6937 leader_election.cc:288] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:23.455111 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c59eea10ec45af8e3a0992f9cc74bf" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:23.456168 5880 raft_consensus.cc:2454] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:23.450939 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c59eea10ec45af8e3a0992f9cc74bf" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:23.444445 6046 raft_consensus.cc:374] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:23.460281 6046 raft_consensus.cc:729] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:23.446424 4837 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 1 got 0
I20260808 13:35:23.470034 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 1de317c48eaa43dca038410840483f52 (DEFAULT_TABLE table=default.Table_2d152c1c176a4bc58ff1f37b2a9bfd0c [id=8f74146d8e454203a91ce42acd6a5233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.471678 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1de317c48eaa43dca038410840483f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.471647 6046 consensus_queue.cc:260] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.473001 4837 catalog_manager.cc:5495] T cc4dc66771804e0cb8991da63eacba0c P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:23.474598 4837 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:23.475086 6999 leader_election.cc:288] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:23.476081 5551 leader_election.cc:341] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.478098 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 1de317c48eaa43dca038410840483f52 (DEFAULT_TABLE table=default.Table_2d152c1c176a4bc58ff1f37b2a9bfd0c [id=8f74146d8e454203a91ce42acd6a5233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.476727 6046 ts_tablet_manager.cc:1428] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.036s user 0.007s sys 0.002s
I20260808 13:35:23.479720 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1de317c48eaa43dca038410840483f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.472146 6033 tablet_bootstrap.cc:654] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.485795 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 7d31516006bc4a65b496dcf363db464a (DEFAULT_TABLE table=default.Table_1c113a710bb14973be6ef54f41e550f8 [id=7929b3dde23940a8a96690fff0077013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.480433 6046 tablet_bootstrap.cc:492] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:23.486655 5346 tablet_service.cc:1467] Processing CreateTablet for tablet 7d31516006bc4a65b496dcf363db464a (DEFAULT_TABLE table=default.Table_1c113a710bb14973be6ef54f41e550f8 [id=7929b3dde23940a8a96690fff0077013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.487353 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d31516006bc4a65b496dcf363db464a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.487923 4837 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 1947f03e107447bb8e23fe5deaf1ea60 (table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a]). Expected version 2 got 0
I20260808 13:35:23.488185 5346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d31516006bc4a65b496dcf363db464a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.491770 4837 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:23.492311 4837 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 973a7afdcbf944aab8f11b3fd3995865 (table default.Table_5c71e9adaaf5477b996f88813566c561 [id=eb20921cee2f48dc86300ce8ce5f52a1]). Expected version 3 got 0
I20260808 13:35:23.496358 5836 tablet_service.cc:1467] Processing CreateTablet for tablet eaa049ef916b44458be43ec4e2ccaf1c (DEFAULT_TABLE table=default.Table_04a268476ef54bb68aadad0d39d6e838 [id=d342034fcd7b45debb1aca898c6b7f68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.497339 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaa049ef916b44458be43ec4e2ccaf1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.501863 6953 raft_consensus.cc:482] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.502348 6953 raft_consensus.cc:504] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.502687 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d (DEFAULT_TABLE table=default.Table_a730d953de56401597f5e28a6edc707a [id=ba2a2d02f38d47a489d40b228fe0af55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.503608 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.504447 6953 leader_election.cc:288] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:23.506142 5357 tablet_service.cc:1467] Processing CreateTablet for tablet eaa049ef916b44458be43ec4e2ccaf1c (DEFAULT_TABLE table=default.Table_04a268476ef54bb68aadad0d39d6e838 [id=d342034fcd7b45debb1aca898c6b7f68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.507416 4837 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_13ca344305b74220b866f3194259e283 [id=16cef8437603476b97a8562edba58a41]). Expected version 3 got 2
I20260808 13:35:23.507659 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaa049ef916b44458be43ec4e2ccaf1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.512398 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d (DEFAULT_TABLE table=default.Table_a730d953de56401597f5e28a6edc707a [id=ba2a2d02f38d47a489d40b228fe0af55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.513592 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.514340 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet b809b185ee9e4c658a050c41c9abc4ad
I20260808 13:35:23.515887 4833 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 257) took 1743 ms. Trace:
I20260808 13:35:23.517323 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet b809b185ee9e4c658a050c41c9abc4ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:23.518076 6585 tablet_replica.cc:330] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:23.518838 6585 raft_consensus.cc:2229] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:23.519501 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 42e36d643850411498c23699835f55a6 (DEFAULT_TABLE table=default.Table_3585f6d6de0e451e9732239606d0e39c [id=5c3825e556e04b17b0437380aa662748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.519742 6585 raft_consensus.cc:2258] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:23.520591 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e36d643850411498c23699835f55a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.521833 6585 ts_tablet_manager.cc:1905] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:23.525480 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet e7616d81aa924933a9b24d44004725fa with delete_type TABLET_DATA_DELETED (replaced by tablet 1b9e8d48c55349058cf80bf7c2e149e7) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:23.525480 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 24) took 4408 ms (client timeout 1000 ms). Trace:
I20260808 13:35:23.526150 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 543debff6da54d26bf62260bf8721ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:23.527190 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a95712de254c13b3b6dc4fe5cb15f7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a27517560d64a3b8da2cf55980e18a8) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:23.516188 4833 rpcz_store.cc:276] 0808 13:35:21.772336 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:21.817969 (+ 45633us) service_pool.cc:227] Handling call
0808 13:35:21.819418 (+ 1449us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:21.819970 (+ 552us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:21.820086 (+ 116us) catalog_manager.cc:2761] Locking tablets
0808 13:35:22.558488 (+738402us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:22.577945 (+ 19457us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:22.578286 (+ 341us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:23.514095 (+935809us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:23.515789 (+ 1694us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:22.562167 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:22.562689 (+ 522us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:22.562727 (+ 38us) write_op.cc:435] Acquired schema lock
0808 13:35:22.562755 (+ 28us) tablet.cc:660] Decoding operations
0808 13:35:22.563188 (+ 433us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:22.563269 (+ 81us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:22.563309 (+ 40us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:22.563663 (+ 354us) tablet.cc:699] Row locks acquired
0808 13:35:22.563694 (+ 31us) write_op.cc:260] PREPARE: finished
0808 13:35:22.563970 (+ 276us) write_op.cc:270] Start()
0808 13:35:22.564116 (+ 146us) write_op.cc:276] Timestamp: P: 1786196122563824 usec, L: 0
0808 13:35:22.564133 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:22.564855 (+ 722us) log.cc:842] Serialized 759 byte log entry
0808 13:35:22.573913 (+ 9058us) op_driver.cc:444] REPLICATION: finished
0808 13:35:22.574804 (+ 891us) write_op.cc:301] APPLY: starting
0808 13:35:22.574866 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:22.574990 (+ 124us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:22.575095 (+ 105us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:22.575446 (+ 351us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:22.575591 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:22.575639 (+ 48us) write_op.cc:312] APPLY: finished
0808 13:35:22.575834 (+ 195us) log.cc:842] Serialized 29 byte log entry
0808 13:35:22.575974 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:22.576043 (+ 69us) write_op.cc:454] Released schema lock
0808 13:35:22.576071 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:22.576254 (+ 183us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:22.576281 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":708,"apply.run_cpu_time_us":1666,"apply.run_wall_time_us":1665,"num_ops":2,"prepare.queue_time_us":360,"prepare.run_cpu_time_us":2980,"prepare.run_wall_time_us":2980,"raft.queue_time_us":1851,"raft.run_cpu_time_us":567,"raft.run_wall_time_us":568,"replication_time_us":9712}]]}
I20260808 13:35:23.529227 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:23.530335 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:08 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:23.530364 4833 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 27) took 1003 ms (client timeout 1000 ms). Trace:
W20260808 13:35:23.530714 4833 rpcz_store.cc:269] 0808 13:35:22.527555 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.530069 (+1002514us) service_pool.cc:214] Skipping call since client already timed out
0808 13:35:23.530308 (+ 239us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260808 13:35:23.532027 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 42e36d643850411498c23699835f55a6 (DEFAULT_TABLE table=default.Table_3585f6d6de0e451e9732239606d0e39c [id=5c3825e556e04b17b0437380aa662748]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:23.525810 4832 rpcz_store.cc:269] 0808 13:35:19.117101 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:19.117199 (+ 98us) service_pool.cc:227] Handling call
0808 13:35:19.122412 (+ 5213us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:23.525279 (+4402867us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:20.634600 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:20.635050 (+ 450us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:20.635075 (+ 25us) write_op.cc:435] Acquired schema lock
0808 13:35:20.635086 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:20.635342 (+ 256us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:20.635384 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:20.635406 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:20.654379 (+ 18973us) tablet.cc:699] Row locks acquired
0808 13:35:20.654405 (+ 26us) write_op.cc:260] PREPARE: finished
0808 13:35:20.654602 (+ 197us) write_op.cc:270] Start()
0808 13:35:20.654736 (+ 134us) write_op.cc:276] Timestamp: P: 1786196120654499 usec, L: 0
0808 13:35:20.654760 (+ 24us) op_driver.cc:331] REPLICATION: starting
0808 13:35:20.655532 (+ 772us) log.cc:842] Serialized 527 byte log entry
0808 13:35:20.680445 (+ 24913us) op_driver.cc:444] REPLICATION: finished
0808 13:35:20.681041 (+ 596us) write_op.cc:301] APPLY: starting
0808 13:35:20.681117 (+ 76us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:20.681377 (+ 260us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:20.681682 (+ 305us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:20.682087 (+ 405us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:20.682379 (+ 292us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:20.682490 (+ 111us) write_op.cc:312] APPLY: finished
0808 13:35:20.682908 (+ 418us) log.cc:842] Serialized 23 byte log entry
0808 13:35:20.683228 (+ 320us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:20.683324 (+ 96us) write_op.cc:454] Released schema lock
0808 13:35:20.683394 (+ 70us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:20.683766 (+ 372us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:20.683823 (+ 57us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35526,"reported_tablets":19,"spinlock_wait_cycles":4111232,"child_traces":[["op",{"apply.queue_time_us":342,"apply.run_cpu_time_us":3047,"apply.run_wall_time_us":3047,"num_ops":1,"prepare.queue_time_us":532939,"prepare.run_cpu_time_us":2738,"prepare.run_wall_time_us":58490,"raft.queue_time_us":12905,"raft.run_cpu_time_us":1028,"raft.run_wall_time_us":1027,"replication_time_us":25587}]]}
W20260808 13:35:23.543864 4842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 42) took 3536 ms (client timeout 1000 ms). Trace:
I20260808 13:35:23.545035 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba5fa465ec04bbb801a821a71a8f87d (DEFAULT_TABLE table=default.Table_291e70645d0a4f24b232391de2e62d9b [id=748566361526422b9dc06be81bcf7286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.546526 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd8f12cda94c4eb9a848f002a4020b" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:23.548383 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet b809b185ee9e4c658a050c41c9abc4ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.550427 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b29786cb404bc2b01473c3b8390a50 with delete_type TABLET_DATA_DELETED (replaced by tablet 362dbd6990e740b39690537af98aa2f8) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.551622 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f8863bc32e49399940912444bcb972 with delete_type TABLET_DATA_DELETED (replaced by tablet 62d68ab73bae4dd9af46e907b1fd198c) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:23.552731 4839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 26) took 2396 ms (client timeout 1000 ms). Trace:
W20260808 13:35:23.553056 4839 rpcz_store.cc:269] 0808 13:35:21.129882 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:21.300737 (+170855us) service_pool.cc:227] Handling call
0808 13:35:23.526490 (+2225753us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":74601,"reported_tablets":20}
W20260808 13:35:23.560494 4838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 35) took 2477 ms (client timeout 1000 ms). Trace:
I20260808 13:35:23.561892 5550 leader_election.cc:302] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:23.563287 4840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 43) took 2404 ms (client timeout 1000 ms). Trace:
W20260808 13:35:23.563587 4840 rpcz_store.cc:269] 0808 13:35:21.158858 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:21.325449 (+166591us) service_pool.cc:227] Handling call
0808 13:35:23.563063 (+2237614us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:22.323781 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:22.324266 (+ 485us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:22.324299 (+ 33us) write_op.cc:435] Acquired schema lock
0808 13:35:22.324328 (+ 29us) tablet.cc:660] Decoding operations
0808 13:35:22.324658 (+ 330us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:22.324715 (+ 57us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:22.324747 (+ 32us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:22.324929 (+ 182us) tablet.cc:699] Row locks acquired
0808 13:35:22.324948 (+ 19us) write_op.cc:260] PREPARE: finished
0808 13:35:22.325081 (+ 133us) write_op.cc:270] Start()
0808 13:35:22.325188 (+ 107us) write_op.cc:276] Timestamp: P: 1786196122324986 usec, L: 0
0808 13:35:22.325210 (+ 22us) op_driver.cc:331] REPLICATION: starting
0808 13:35:22.325852 (+ 642us) log.cc:842] Serialized 527 byte log entry
0808 13:35:22.539890 (+214038us) op_driver.cc:444] REPLICATION: finished
0808 13:35:22.548468 (+ 8578us) write_op.cc:301] APPLY: starting
0808 13:35:22.548570 (+ 102us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:22.548703 (+ 133us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:22.548818 (+ 115us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:22.549063 (+ 245us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:22.549261 (+ 198us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:22.549310 (+ 49us) write_op.cc:312] APPLY: finished
0808 13:35:22.549607 (+ 297us) log.cc:842] Serialized 23 byte log entry
0808 13:35:22.549776 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:22.549866 (+ 90us) write_op.cc:454] Released schema lock
0808 13:35:22.549901 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:22.550131 (+ 230us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:22.550153 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":8313,"apply.run_cpu_time_us":1869,"apply.run_wall_time_us":1869,"num_ops":1,"prepare.queue_time_us":127,"prepare.run_cpu_time_us":2300,"prepare.run_wall_time_us":2301,"replication_time_us":214451}]]}
I20260808 13:35:23.567369 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 1de317c48eaa43dca038410840483f52 (DEFAULT_TABLE table=default.Table_2d152c1c176a4bc58ff1f37b2a9bfd0c [id=8f74146d8e454203a91ce42acd6a5233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.567690 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e36d643850411498c23699835f55a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.568390 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1de317c48eaa43dca038410840483f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.569365 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 7d31516006bc4a65b496dcf363db464a (DEFAULT_TABLE table=default.Table_1c113a710bb14973be6ef54f41e550f8 [id=7929b3dde23940a8a96690fff0077013]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:23.570112 4836 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 34) took 3522 ms (client timeout 1000 ms). Trace:
I20260808 13:35:23.570444 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d31516006bc4a65b496dcf363db464a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.571358 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d (DEFAULT_TABLE table=default.Table_a730d953de56401597f5e28a6edc707a [id=ba2a2d02f38d47a489d40b228fe0af55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.572261 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.572743 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet b809b185ee9e4c658a050c41c9abc4ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:23.575150 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba5fa465ec04bbb801a821a71a8f87d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.576377 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd8f12cda94c4eb9a848f002a4020b" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:23.576956 5642 raft_consensus.cc:2454] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:23.578478 5772 leader_election.cc:302] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:23.579247 6953 raft_consensus.cc:2781] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:23.579571 6953 raft_consensus.cc:482] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:23.579855 6953 raft_consensus.cc:3037] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:23.580338 4841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 25) took 3482 ms (client timeout 1000 ms). Trace:
I20260808 13:35:23.581015 4835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 196d80d806b0432cacf2ca6aa0da8c6b
W20260808 13:35:23.580653 4841 rpcz_store.cc:269] 0808 13:35:20.097390 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.151470 (+ 54080us) service_pool.cc:227] Handling call
0808 13:35:23.580132 (+3428662us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16353,"reported_tablets":20}
I20260808 13:35:23.588482 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 42e36d643850411498c23699835f55a6 (DEFAULT_TABLE table=default.Table_3585f6d6de0e451e9732239606d0e39c [id=5c3825e556e04b17b0437380aa662748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.589630 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e36d643850411498c23699835f55a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.591176 5606 tablet_service.cc:1467] Processing CreateTablet for tablet eaa049ef916b44458be43ec4e2ccaf1c (DEFAULT_TABLE table=default.Table_04a268476ef54bb68aadad0d39d6e838 [id=d342034fcd7b45debb1aca898c6b7f68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.584584 6937 raft_consensus.cc:2781] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:23.599299 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d8018bf8ff843d8818fbba9473acf47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.599365 6937 raft_consensus.cc:482] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:23.599699 6937 raft_consensus.cc:3037] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.600507 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 543debff6da54d26bf62260bf8721ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.601472 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet e7616d81aa924933a9b24d44004725fa with delete_type TABLET_DATA_DELETED (replaced by tablet 1b9e8d48c55349058cf80bf7c2e149e7) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.602388 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc9a6ec9a86431fb46e50fcaa658d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.603331 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d13cbde6844079c2d1eb80732bb1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.604440 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d492406c8a4c4186131e02a1559cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.582479 4835 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 258) took 1796 ms. Trace:
I20260808 13:35:23.607722 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet d2ac4c366363468aa62b73e0f1e04532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:23.608544 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fb3e099f7f4544b3c100070f0d14ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:23.584342 6901 raft_consensus.cc:482] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.605762 4835 rpcz_store.cc:276] 0808 13:35:21.785683 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:21.952819 (+167136us) service_pool.cc:227] Handling call
0808 13:35:22.054059 (+101240us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:22.054586 (+ 527us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:22.054750 (+ 164us) catalog_manager.cc:2761] Locking tablets
0808 13:35:23.425644 (+1370894us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:23.554718 (+129074us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:23.580723 (+ 26005us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:23.580838 (+ 115us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:23.582418 (+ 1580us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:23.449257 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:23.450259 (+ 1002us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:23.450323 (+ 64us) write_op.cc:435] Acquired schema lock
0808 13:35:23.450380 (+ 57us) tablet.cc:660] Decoding operations
0808 13:35:23.451352 (+ 972us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:23.451448 (+ 96us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:23.451505 (+ 57us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:23.458037 (+ 6532us) tablet.cc:699] Row locks acquired
0808 13:35:23.458105 (+ 68us) write_op.cc:260] PREPARE: finished
0808 13:35:23.458370 (+ 265us) write_op.cc:270] Start()
0808 13:35:23.458545 (+ 175us) write_op.cc:276] Timestamp: P: 1786196123458261 usec, L: 0
0808 13:35:23.458567 (+ 22us) op_driver.cc:331] REPLICATION: starting
0808 13:35:23.460897 (+ 2330us) log.cc:842] Serialized 725 byte log entry
0808 13:35:23.505227 (+ 44330us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b58000c05b0. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c1a9834 00007f227c1ae771 00007f227c1ae6f9 00007f227c1ae689 00007f227c1ae651 00007f227c1ad81d 0000000000580784 00000000005805b9 00007f22752da6b6 00007f22752ddb11
0808 13:35:23.551592 (+ 46365us) op_driver.cc:444] REPLICATION: finished
0808 13:35:23.551908 (+ 316us) write_op.cc:301] APPLY: starting
0808 13:35:23.551974 (+ 66us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:23.552091 (+ 117us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:23.552117 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:23.552440 (+ 323us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:23.552613 (+ 173us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:23.552703 (+ 90us) write_op.cc:312] APPLY: finished
0808 13:35:23.552979 (+ 276us) log.cc:842] Serialized 29 byte log entry
0808 13:35:23.553188 (+ 209us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:23.553275 (+ 87us) write_op.cc:454] Released schema lock
0808 13:35:23.553320 (+ 45us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:23.553557 (+ 237us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:23.553594 (+ 37us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":124,"apply.run_cpu_time_us":1932,"apply.run_wall_time_us":1932,"num_ops":2,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":14397,"prepare.run_wall_time_us":102099,"raft.queue_time_us":43598,"raft.run_cpu_time_us":1990,"raft.run_wall_time_us":42615,"replication_time_us":92940,"spinlock_wait_cycles":93949824,"thread_start_us":88331,"threads_started":1}]]}
I20260808 13:35:23.609306 6901 raft_consensus.cc:504] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.615135 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a95712de254c13b3b6dc4fe5cb15f7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a27517560d64a3b8da2cf55980e18a8) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:23.616039 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:23.618052 6796 raft_consensus.cc:482] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.618525 6796 raft_consensus.cc:504] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.622762 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaa049ef916b44458be43ec4e2ccaf1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.619122 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_aa07254565ac49d4aadd887511d6838b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:23.624841 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa07254565ac49d4aadd887511d6838b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:23.626874 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 2173e97590be43f9a24f55689d796269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:23.627553 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:23.628583 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet 2173e97590be43f9a24f55689d796269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:23.630263 6901 leader_election.cc:288] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:23.560794 4838 rpcz_store.cc:269] 0808 13:35:21.083068 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:21.282332 (+199264us) service_pool.cc:227] Handling call
0808 13:35:21.294372 (+ 12040us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:23.560309 (+2265937us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:21.798736 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:21.799083 (+ 347us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:21.799104 (+ 21us) write_op.cc:435] Acquired schema lock
0808 13:35:21.799114 (+ 10us) tablet.cc:660] Decoding operations
0808 13:35:21.799349 (+ 235us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:21.799383 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:21.799404 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:21.799536 (+ 132us) tablet.cc:699] Row locks acquired
0808 13:35:21.799548 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:21.799647 (+ 99us) write_op.cc:270] Start()
0808 13:35:21.799723 (+ 76us) write_op.cc:276] Timestamp: P: 1786196121799567 usec, L: 0
0808 13:35:21.799737 (+ 14us) op_driver.cc:331] REPLICATION: starting
0808 13:35:21.800179 (+ 442us) log.cc:842] Serialized 527 byte log entry
0808 13:35:21.819100 (+ 18921us) op_driver.cc:444] REPLICATION: finished
0808 13:35:21.824168 (+ 5068us) write_op.cc:301] APPLY: starting
0808 13:35:21.824244 (+ 76us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:21.824333 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:21.824425 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:21.824577 (+ 152us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:21.824695 (+ 118us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:21.824733 (+ 38us) write_op.cc:312] APPLY: finished
0808 13:35:21.824894 (+ 161us) log.cc:842] Serialized 23 byte log entry
0808 13:35:21.825473 (+ 579us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:21.825534 (+ 61us) write_op.cc:454] Released schema lock
0808 13:35:21.825562 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:21.825719 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:21.825741 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2018,"reported_tablets":13,"spinlock_wait_cycles":16977152,"child_traces":[["op",{"apply.queue_time_us":4925,"apply.run_cpu_time_us":1298,"apply.run_wall_time_us":1738,"mutex_wait_us":35546,"num_ops":1,"prepare.queue_time_us":35665,"prepare.run_cpu_time_us":1682,"prepare.run_wall_time_us":1814,"raft.queue_time_us":74,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":19275}]]}
I20260808 13:35:23.631850 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd8f12cda94c4eb9a848f002a4020b" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:23.631982 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba5fa465ec04bbb801a821a71a8f87d (DEFAULT_TABLE table=default.Table_291e70645d0a4f24b232391de2e62d9b [id=748566361526422b9dc06be81bcf7286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.632342 6796 leader_election.cc:288] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:23.632213 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d56fc9f7ea3a43f98fe5a3057b51dbac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:23.633165 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba5fa465ec04bbb801a821a71a8f87d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:23.633797 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d56fc9f7ea3a43f98fe5a3057b51dbac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:23.634841 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b29786cb404bc2b01473c3b8390a50 with delete_type TABLET_DATA_DELETED (replaced by tablet 362dbd6990e740b39690537af98aa2f8) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:23.570528 4836 rpcz_store.cc:269] 0808 13:35:20.047214 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.121533 (+ 74319us) service_pool.cc:227] Handling call
0808 13:35:23.569882 (+3448349us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":12}
I20260808 13:35:23.635536 7014 tablet_replica.cc:330] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:23.635638 4836 master_service.cc:946] ReplaceTablet: received request to replace tablet 86c2b34c95244472beec43643e60ff5f from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:23.636354 7014 raft_consensus.cc:2229] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 13:35:23.544174 4842 rpcz_store.cc:269] 0808 13:35:20.007095 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:20.023699 (+ 16604us) service_pool.cc:227] Handling call
0808 13:35:20.115607 (+ 91908us) spinlock_profiling.cc:243] Waited 88 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:23.543679 (+3428072us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:21.137927 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:21.138339 (+ 412us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:21.138367 (+ 28us) write_op.cc:435] Acquired schema lock
0808 13:35:21.138380 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:21.138694 (+ 314us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:21.138737 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:21.138759 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:21.138918 (+ 159us) tablet.cc:699] Row locks acquired
0808 13:35:21.138933 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:21.139061 (+ 128us) write_op.cc:270] Start()
0808 13:35:21.139164 (+ 103us) write_op.cc:276] Timestamp: P: 1786196121138985 usec, L: 0
0808 13:35:21.139185 (+ 21us) op_driver.cc:331] REPLICATION: starting
0808 13:35:21.139723 (+ 538us) log.cc:842] Serialized 356 byte log entry
0808 13:35:21.168023 (+ 28300us) op_driver.cc:444] REPLICATION: finished
0808 13:35:21.168348 (+ 325us) write_op.cc:301] APPLY: starting
0808 13:35:21.168405 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:21.168502 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:21.168729 (+ 227us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:21.169047 (+ 318us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:21.169357 (+ 310us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:21.169451 (+ 94us) write_op.cc:312] APPLY: finished
0808 13:35:21.169758 (+ 307us) log.cc:842] Serialized 23 byte log entry
0808 13:35:21.170005 (+ 247us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:21.170091 (+ 86us) write_op.cc:454] Released schema lock
0808 13:35:21.170178 (+ 87us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:21.170450 (+ 272us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:21.170507 (+ 57us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":19,"spinlock_wait_cycles":202306816,"child_traces":[["op",{"apply.queue_time_us":183,"apply.run_cpu_time_us":2408,"apply.run_wall_time_us":2409,"num_ops":1,"prepare.queue_time_us":424039,"prepare.run_cpu_time_us":2153,"prepare.run_wall_time_us":23376,"raft.queue_time_us":20191,"raft.run_cpu_time_us":637,"raft.run_wall_time_us":637,"replication_time_us":28745}]]}
I20260808 13:35:23.636793 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:22 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:23.636868 7014 raft_consensus.cc:2258] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:23.637437 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_9769e23d59f2498e925cd703900ecbb3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:23.639005 5637 raft_consensus.cc:2454] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:23.645923 4835 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_3e6fd22e0fd24df4b30ca0044f1853bd" } new_table_name: "default.Table_321b4eb3dd024925af1cb354d3fe1567" modify_external_catalogs: true
W20260808 13:35:23.648494 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9769e23d59f2498e925cd703900ecbb3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:23.651429 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.652537 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 2173e97590be43f9a24f55689d796269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.657317 6046 tablet_bootstrap.cc:654] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:23.657522 6937 raft_consensus.cc:504] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.660127 5285 leader_election.cc:302] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:23.661422 6999 raft_consensus.cc:2781] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:23.661919 6999 raft_consensus.cc:686] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:23.662649 6999 consensus_queue.cc:237] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.622422 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_dae35fadbf624671ad716102b5182446"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:23.671841 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dae35fadbf624671ad716102b5182446 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:23.673931 4821 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_d30a1f7564df4e6085a1858c6f3924fc [id=4296a36ca9b74888aeccfe22e3970de6], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f66b38e83dab4bf1a3d2d1e088a96988 consensus state has no leader
I20260808 13:35:23.675077 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 7454bf27bc1a45d2af0d8f897b2f9f97 (DEFAULT_TABLE table=default.Table_0616044c71e04d1a83b3ffd25b84f54e [id=71c485130d814412bd042eb5012ba511]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.676673 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7454bf27bc1a45d2af0d8f897b2f9f97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.658802 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c083c8e19d4279af600670f5d1d78d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:23.680845 5642 raft_consensus.cc:2454] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:23.673257 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 7454bf27bc1a45d2af0d8f897b2f9f97 (DEFAULT_TABLE table=default.Table_0616044c71e04d1a83b3ffd25b84f54e [id=71c485130d814412bd042eb5012ba511]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.684031 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7454bf27bc1a45d2af0d8f897b2f9f97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.684954 5337 tablet_service.cc:1467] Processing CreateTablet for tablet b8e3245c8ba54f69bc329a5b75115127 (DEFAULT_TABLE table=default.Table_fd822a88bb39430eb487b2bdc43e86fb [id=093c441116e34908b83de704bbb58ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.675666 5600 tablet_service.cc:1467] Processing CreateTablet for tablet b8e3245c8ba54f69bc329a5b75115127 (DEFAULT_TABLE table=default.Table_fd822a88bb39430eb487b2bdc43e86fb [id=093c441116e34908b83de704bbb58ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.676385 7014 ts_tablet_manager.cc:1905] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:23.686088 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e3245c8ba54f69bc329a5b75115127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.686816 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e3245c8ba54f69bc329a5b75115127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.700105 5609 tablet_service.cc:1467] Processing CreateTablet for tablet b1c41483119a442eb46816aa34f7f319 (DEFAULT_TABLE table=default.Table_a16be245c9864b969707b18ee86278d0 [id=8f151f8823464520ae72942a6c79d8bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.701493 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1c41483119a442eb46816aa34f7f319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.741482 5345 tablet_service.cc:1467] Processing CreateTablet for tablet b1c41483119a442eb46816aa34f7f319 (DEFAULT_TABLE table=default.Table_a16be245c9864b969707b18ee86278d0 [id=8f151f8823464520ae72942a6c79d8bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.741824 5614 tablet_service.cc:1467] Processing CreateTablet for tablet ba39706cff65404485959865fff1772a (DEFAULT_TABLE table=default.Table_fc7cb305e35541c781c3e3790cf8a0af [id=b885c5ab382c45809d6f896ebf8771bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.741719 5616 tablet_service.cc:1467] Processing CreateTablet for tablet fc3e9c6770fd402290b45eed85de6f6f (DEFAULT_TABLE table=default.Table_bf8907064282450eb869d5bb8ec9f1c1 [id=07969203fd174637a09f5b8e83bc7402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.742952 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1c41483119a442eb46816aa34f7f319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.743304 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba39706cff65404485959865fff1772a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.744108 6937 leader_election.cc:288] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:23.747453 5285 leader_election.cc:302] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:23.748478 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc3e9c6770fd402290b45eed85de6f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.748768 6913 consensus_queue.cc:1035] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:23.750517 6913 raft_consensus.cc:2781] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:23.750878 6913 raft_consensus.cc:482] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:23.751209 6913 raft_consensus.cc:3037] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:23.757711 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b29786cb404bc2b01473c3b8390a50 with delete_type TABLET_DATA_DELETED (replaced by tablet 362dbd6990e740b39690537af98aa2f8) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:23.765389 5336 tablet_service.cc:1467] Processing CreateTablet for tablet fc3e9c6770fd402290b45eed85de6f6f (DEFAULT_TABLE table=default.Table_bf8907064282450eb869d5bb8ec9f1c1 [id=07969203fd174637a09f5b8e83bc7402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.766571 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc3e9c6770fd402290b45eed85de6f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.767663 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:22 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:23.780134 5335 tablet_service.cc:1467] Processing CreateTablet for tablet de6800eceb96475284fdbffb4d23e5e3 (DEFAULT_TABLE table=default.Table_9fa5c49027684fe6a85a46895e121932 [id=7fff2277bc2f4edd9aeccc15d58333c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.781286 5335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6800eceb96475284fdbffb4d23e5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.808223 7014 ts_tablet_manager.cc:1918] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:23.809937 5334 tablet_service.cc:1467] Processing CreateTablet for tablet 538d9a883e9d46668fde30876352d20e (DEFAULT_TABLE table=default.Table_114737be7a5c48089f667a5f02f76b5e [id=e3e121c36ff44ea0a3a98185c8cce381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.811802 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9e8d48c55349058cf80bf7c2e149e7 (DEFAULT_TABLE table=default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.812083 5607 tablet_service.cc:1467] Processing CreateTablet for tablet de6800eceb96475284fdbffb4d23e5e3 (DEFAULT_TABLE table=default.Table_9fa5c49027684fe6a85a46895e121932 [id=7fff2277bc2f4edd9aeccc15d58333c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.812813 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b9e8d48c55349058cf80bf7c2e149e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.813141 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6800eceb96475284fdbffb4d23e5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.817123 7013 tablet_replica.cc:330] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:23.818574 7013 raft_consensus.cc:2229] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:23.811383 7014 log.cc:1196] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/b809b185ee9e4c658a050c41c9abc4ad
I20260808 13:35:23.823623 7014 ts_tablet_manager.cc:1939] T b809b185ee9e4c658a050c41c9abc4ad P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:23.824710 5355 tablet_service.cc:1467] Processing CreateTablet for tablet ba39706cff65404485959865fff1772a (DEFAULT_TABLE table=default.Table_fc7cb305e35541c781c3e3790cf8a0af [id=b885c5ab382c45809d6f896ebf8771bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.829404 5332 tablet_service.cc:1467] Processing CreateTablet for tablet d09eb729339e47eb8db88eb6509daab4 (DEFAULT_TABLE table=default.Table_03b5de95197845c1bc7ab06aec3d983d [id=af41016671a644dcaacf0b2602d83672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.832526 6026 tablet_bootstrap.cc:492] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:23.832974 6026 ts_tablet_manager.cc:1397] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.523s user 0.008s sys 0.003s
I20260808 13:35:23.837077 6901 raft_consensus.cc:482] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:23.837469 6901 raft_consensus.cc:504] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.840421 7014 tablet_replica.cc:330] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:23.843781 6901 leader_election.cc:288] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:23.855676 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 37495b29698940edb2924eb612e94bd7 (DEFAULT_TABLE table=default.Table_c3cd224a54354169a0cba0ad45c3257e [id=ed0521e8ecae4c609b2a7426e5a565c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.858873 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd8f12cda94c4eb9a848f002a4020b" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:23.868959 5551 leader_election.cc:341] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.883601 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:23.887151 7014 raft_consensus.cc:2229] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:23.890084 7014 pending_rounds.cc:62] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Trying to abort 1 pending ops.
I20260808 13:35:23.890374 7014 pending_rounds.cc:68] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259318776066048 op_type: NO_OP noop_request { }
I20260808 13:35:23.893596 7014 raft_consensus.cc:2866] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:35:23.894002 7014 raft_consensus.cc:2258] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:23.894845 5614 tablet_service.cc:1467] Processing CreateTablet for tablet d09eb729339e47eb8db88eb6509daab4 (DEFAULT_TABLE table=default.Table_03b5de95197845c1bc7ab06aec3d983d [id=af41016671a644dcaacf0b2602d83672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.895201 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet b809b185ee9e4c658a050c41c9abc4ad (table default.Table_1680269ff16246c382d6bccf44f8d52e [id=a15e38df2aee420fb9345cabd03dc842]) successfully deleted
I20260808 13:35:23.895941 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d09eb729339e47eb8db88eb6509daab4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:23.896032 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet d2ac4c366363468aa62b73e0f1e04532 because the tablet was not found. No further retry: Not found: Tablet not found: d2ac4c366363468aa62b73e0f1e04532
W20260808 13:35:23.896875 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 81fb3e099f7f4544b3c100070f0d14ef because the tablet was not found. No further retry: Not found: Tablet not found: 81fb3e099f7f4544b3c100070f0d14ef
W20260808 13:35:23.897642 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet e2a95712de254c13b3b6dc4fe5cb15f7 because the tablet was not found. No further retry: Not found: Tablet not found: e2a95712de254c13b3b6dc4fe5cb15f7
I20260808 13:35:23.894887 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 37495b29698940edb2924eb612e94bd7 (DEFAULT_TABLE table=default.Table_c3cd224a54354169a0cba0ad45c3257e [id=ed0521e8ecae4c609b2a7426e5a565c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.900151 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37495b29698940edb2924eb612e94bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.907155 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9e8d48c55349058cf80bf7c2e149e7 (DEFAULT_TABLE table=default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.908373 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b9e8d48c55349058cf80bf7c2e149e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.925181 5607 tablet_service.cc:1467] Processing CreateTablet for tablet 538d9a883e9d46668fde30876352d20e (DEFAULT_TABLE table=default.Table_114737be7a5c48089f667a5f02f76b5e [id=e3e121c36ff44ea0a3a98185c8cce381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:23.926304 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538d9a883e9d46668fde30876352d20e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:23.926647 7030 consensus_queue.cc:1159] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260808 13:35:23.929586 7014 ts_tablet_manager.cc:1905] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:23.931861 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c59eea10ec45af8e3a0992f9cc74bf" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:23.949564 5551 leader_election.cc:341] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:23.952457 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:23.953142 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 1947f03e107447bb8e23fe5deaf1ea60 (table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a]). Expected version 2 got 0
I20260808 13:35:23.956928 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:23.958025 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:23.992604 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37495b29698940edb2924eb612e94bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:23.827673 6913 raft_consensus.cc:504] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:23.863399 7013 raft_consensus.cc:2258] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:23.982477 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:24.023021 6026 raft_consensus.cc:348] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.023764 6026 raft_consensus.cc:374] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:24.024010 6026 raft_consensus.cc:729] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:24.033512 6585 ts_tablet_manager.cc:1918] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:24.033943 6585 log.cc:1196] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/b809b185ee9e4c658a050c41c9abc4ad
I20260808 13:35:24.035287 6026 consensus_queue.cc:260] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.062408 4838 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 262) took 2042 ms. Trace:
I20260808 13:35:24.062785 4838 rpcz_store.cc:276] 0808 13:35:22.019992 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.631756 (+1611764us) service_pool.cc:227] Handling call
0808 13:35:23.680492 (+ 48736us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:23.682133 (+ 1641us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:24.061772 (+379639us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:24.062079 (+ 307us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:24.062341 (+ 262us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:23.829558 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:23.830024 (+ 466us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:23.830053 (+ 29us) write_op.cc:435] Acquired schema lock
0808 13:35:23.830071 (+ 18us) tablet.cc:660] Decoding operations
0808 13:35:23.887128 (+ 57057us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:23.887186 (+ 58us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:23.887224 (+ 38us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:23.887503 (+ 279us) tablet.cc:699] Row locks acquired
0808 13:35:23.887529 (+ 26us) write_op.cc:260] PREPARE: finished
0808 13:35:23.887668 (+ 139us) write_op.cc:270] Start()
0808 13:35:23.887793 (+ 125us) write_op.cc:276] Timestamp: P: 1786196123887589 usec, L: 0
0808 13:35:23.887829 (+ 36us) op_driver.cc:331] REPLICATION: starting
0808 13:35:23.888488 (+ 659us) log.cc:842] Serialized 433 byte log entry
0808 13:35:24.059177 (+170689us) op_driver.cc:444] REPLICATION: finished
0808 13:35:24.059440 (+ 263us) write_op.cc:301] APPLY: starting
0808 13:35:24.059496 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:24.059607 (+ 111us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:24.059696 (+ 89us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:24.060191 (+ 495us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:24.060340 (+ 149us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:24.060387 (+ 47us) write_op.cc:312] APPLY: finished
0808 13:35:24.060574 (+ 187us) log.cc:842] Serialized 29 byte log entry
0808 13:35:24.060715 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.060780 (+ 65us) write_op.cc:454] Released schema lock
0808 13:35:24.060807 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:24.060990 (+ 183us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.061013 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":106,"apply.run_cpu_time_us":1730,"apply.run_wall_time_us":1732,"num_ops":2,"prepare.queue_time_us":146021,"prepare.run_cpu_time_us":2834,"prepare.run_wall_time_us":60587,"raft.queue_time_us":139,"raft.run_cpu_time_us":634,"raft.run_wall_time_us":4965,"replication_time_us":171260,"spinlock_wait_cycles":3328}]]}
I20260808 13:35:24.061422 6585 ts_tablet_manager.cc:1939] T b809b185ee9e4c658a050c41c9abc4ad P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
W20260808 13:35:24.083252 5771 leader_election.cc:341] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:24.086594 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6e6ce417cc45fd90f5858c30d37a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:24.036131 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c59eea10ec45af8e3a0992f9cc74bf" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:24.015822 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538d9a883e9d46668fde30876352d20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:24.096691 5879 raft_consensus.cc:3037] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:24.100909 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fb3e099f7f4544b3c100070f0d14ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:24.103736 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 2173e97590be43f9a24f55689d796269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:24.104285 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d8018bf8ff843d8818fbba9473acf47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:24.108949 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba5fa465ec04bbb801a821a71a8f87d (DEFAULT_TABLE table=default.Table_291e70645d0a4f24b232391de2e62d9b [id=748566361526422b9dc06be81bcf7286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.109961 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba5fa465ec04bbb801a821a71a8f87d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:24.106199 6033 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.607s user 0.013s sys 0.049s
W20260808 13:35:24.117165 6033 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.618s user 0.013s sys 0.049s
I20260808 13:35:24.118446 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:22 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:24.113199 5772 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.000s sys 0.013s
W20260808 13:35:24.122099 5772 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.519s user 0.000s sys 0.014s
W20260808 13:35:24.126026 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 522752us
I20260808 13:35:24.130337 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet d2ac4c366363468aa62b73e0f1e04532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:24.130542 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 7454bf27bc1a45d2af0d8f897b2f9f97 (DEFAULT_TABLE table=default.Table_0616044c71e04d1a83b3ffd25b84f54e [id=71c485130d814412bd042eb5012ba511]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.131598 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7454bf27bc1a45d2af0d8f897b2f9f97. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:24.132762 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 46257.6us
I20260808 13:35:24.134953 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c083c8e19d4279af600670f5d1d78d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:24.136039 6901 raft_consensus.cc:482] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:24.136528 6901 raft_consensus.cc:504] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.140671 4842 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 264) took 1985 ms. Trace:
I20260808 13:35:24.142923 6901 leader_election.cc:288] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:24.141021 4842 rpcz_store.cc:276] 0808 13:35:22.155056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.637013 (+1481957us) service_pool.cc:227] Handling call
0808 13:35:23.683560 (+ 46547us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:23.686587 (+ 3027us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:24.139981 (+453394us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:24.140343 (+ 362us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:24.140608 (+ 265us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:23.890289 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:23.890724 (+ 435us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:23.890751 (+ 27us) write_op.cc:435] Acquired schema lock
0808 13:35:23.890769 (+ 18us) tablet.cc:660] Decoding operations
0808 13:35:23.891113 (+ 344us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:23.891150 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:23.891175 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:23.891381 (+ 206us) tablet.cc:699] Row locks acquired
0808 13:35:23.891398 (+ 17us) write_op.cc:260] PREPARE: finished
0808 13:35:23.891506 (+ 108us) write_op.cc:270] Start()
0808 13:35:23.891590 (+ 84us) write_op.cc:276] Timestamp: P: 1786196123891438 usec, L: 0
0808 13:35:23.891609 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:23.892101 (+ 492us) log.cc:842] Serialized 433 byte log entry
0808 13:35:24.137513 (+245412us) op_driver.cc:444] REPLICATION: finished
0808 13:35:24.138406 (+ 893us) write_op.cc:301] APPLY: starting
0808 13:35:24.138465 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:24.138567 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:24.138713 (+ 146us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:24.139145 (+ 432us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:24.139288 (+ 143us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:24.139333 (+ 45us) write_op.cc:312] APPLY: finished
0808 13:35:24.139539 (+ 206us) log.cc:842] Serialized 29 byte log entry
0808 13:35:24.139678 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.139746 (+ 68us) write_op.cc:454] Released schema lock
0808 13:35:24.139781 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:24.139965 (+ 184us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.139988 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":734,"apply.run_cpu_time_us":1732,"apply.run_wall_time_us":1740,"num_ops":2,"prepare.queue_time_us":202381,"prepare.run_cpu_time_us":2188,"prepare.run_wall_time_us":170079,"raft.queue_time_us":71,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":179,"replication_time_us":245831}]]}
I20260808 13:35:24.148878 4842 master_service.cc:946] ReplaceTablet: received request to replace tablet 973a7afdcbf944aab8f11b3fd3995865 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:24.157788 4840 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 260) took 2279 ms. Trace:
I20260808 13:35:24.158144 4840 rpcz_store.cc:276] 0808 13:35:21.878702 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.566184 (+1687482us) service_pool.cc:227] Handling call
0808 13:35:23.625311 (+ 59127us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:23.689015 (+ 63704us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:24.157102 (+468087us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:24.157456 (+ 354us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:24.157714 (+ 258us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:24.060646 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:24.061086 (+ 440us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:24.061111 (+ 25us) write_op.cc:435] Acquired schema lock
0808 13:35:24.061127 (+ 16us) tablet.cc:660] Decoding operations
0808 13:35:24.061529 (+ 402us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:24.061572 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:24.061595 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:24.061819 (+ 224us) tablet.cc:699] Row locks acquired
0808 13:35:24.061835 (+ 16us) write_op.cc:260] PREPARE: finished
0808 13:35:24.061945 (+ 110us) write_op.cc:270] Start()
0808 13:35:24.062026 (+ 81us) write_op.cc:276] Timestamp: P: 1786196124061861 usec, L: 0
0808 13:35:24.062059 (+ 33us) op_driver.cc:331] REPLICATION: starting
0808 13:35:24.137219 (+ 75160us) log.cc:842] Serialized 433 byte log entry
0808 13:35:24.147822 (+ 10603us) op_driver.cc:444] REPLICATION: finished
0808 13:35:24.154930 (+ 7108us) write_op.cc:301] APPLY: starting
0808 13:35:24.155012 (+ 82us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:24.155141 (+ 129us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:24.155261 (+ 120us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:24.155799 (+ 538us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:24.156078 (+ 279us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:24.156152 (+ 74us) write_op.cc:312] APPLY: finished
0808 13:35:24.156483 (+ 331us) log.cc:842] Serialized 29 byte log entry
0808 13:35:24.156699 (+ 216us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.156802 (+ 103us) write_op.cc:454] Released schema lock
0808 13:35:24.156858 (+ 56us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:24.157129 (+ 271us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.157159 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6855,"apply.run_cpu_time_us":2449,"apply.run_wall_time_us":2437,"num_ops":2,"prepare.queue_time_us":370417,"prepare.run_cpu_time_us":2285,"prepare.run_wall_time_us":78387,"replication_time_us":85623}]]}
I20260808 13:35:24.163269 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_41272f24d0a5431b9f718128ad86cb81" } modify_external_catalogs: true
I20260808 13:35:24.164175 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_41272f24d0a5431b9f718128ad86cb81" } modify_external_catalogs: true
I20260808 13:35:24.171144 6033 tablet_bootstrap.cc:492] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:24.171687 6033 ts_tablet_manager.cc:1397] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.777s user 0.015s sys 0.061s
I20260808 13:35:24.178756 4836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 86c2b34c95244472beec43643e60ff5f
I20260808 13:35:24.180148 4836 catalog_manager.cc:6565] ReplaceTablet: tablet 86c2b34c95244472beec43643e60ff5f deleted and replaced by tablet b314734698ed423ba9cf4134db5846ec
I20260808 13:35:24.179905 6033 raft_consensus.cc:348] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.180512 4836 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60270 (request call id 263) took 2121 ms. Trace:
I20260808 13:35:24.180585 6033 raft_consensus.cc:374] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:24.180822 6033 raft_consensus.cc:729] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:24.181396 6033 consensus_queue.cc:260] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.180773 4836 rpcz_store.cc:276] 0808 13:35:22.058959 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.635425 (+1576466us) service_pool.cc:227] Handling call
0808 13:35:24.180452 (+545027us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:24.139101 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:24.139530 (+ 429us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:24.139557 (+ 27us) write_op.cc:435] Acquired schema lock
0808 13:35:24.139574 (+ 17us) tablet.cc:660] Decoding operations
0808 13:35:24.139939 (+ 365us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:24.139975 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:24.140007 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:24.140231 (+ 224us) tablet.cc:699] Row locks acquired
0808 13:35:24.140252 (+ 21us) write_op.cc:260] PREPARE: finished
0808 13:35:24.140359 (+ 107us) write_op.cc:270] Start()
0808 13:35:24.140451 (+ 92us) write_op.cc:276] Timestamp: P: 1786196124140329 usec, L: 0
0808 13:35:24.140488 (+ 37us) op_driver.cc:331] REPLICATION: starting
0808 13:35:24.144819 (+ 4331us) log.cc:842] Serialized 622 byte log entry
0808 13:35:24.171703 (+ 26884us) op_driver.cc:444] REPLICATION: finished
0808 13:35:24.174889 (+ 3186us) write_op.cc:301] APPLY: starting
0808 13:35:24.175019 (+ 130us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:24.175332 (+ 313us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:24.175506 (+ 174us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:24.176198 (+ 692us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:24.176703 (+ 505us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:24.176766 (+ 63us) write_op.cc:312] APPLY: finished
0808 13:35:24.177081 (+ 315us) log.cc:842] Serialized 29 byte log entry
0808 13:35:24.177895 (+ 814us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.178009 (+ 114us) write_op.cc:454] Released schema lock
0808 13:35:24.178042 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2938,"num_ops":2,"prepare.queue_time_us":446346,"prepare.run_cpu_time_us":2165,"prepare.run_wall_time_us":5963,"replication_time_us":31141}]]}
I20260808 13:35:24.187927 4836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_203fa9f53a2345519f4a5825d2661b97" } modify_external_catalogs: true
I20260808 13:35:24.188515 4836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_203fa9f53a2345519f4a5825d2661b97" } modify_external_catalogs: true
I20260808 13:35:24.159751 4838 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_31cc5d89c86849ae876aa34027918bbc" } new_table_name: "default.Table_7a1963a007954f8ca1d330f9d1936dd3" modify_external_catalogs: true
W20260808 13:35:24.037132 5356 consensus_meta.cc:287] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c: Time spent flushing consensus metadata: real 0.595s user 0.008s sys 0.001s
I20260808 13:35:24.201601 6026 ts_tablet_manager.cc:1428] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.368s user 0.005s sys 0.002s
I20260808 13:35:24.220897 6046 tablet_bootstrap.cc:492] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:24.221339 6046 ts_tablet_manager.cc:1397] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.741s user 0.014s sys 0.060s
I20260808 13:35:24.233418 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d09eb729339e47eb8db88eb6509daab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:24.259104 6046 raft_consensus.cc:348] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.259891 6046 raft_consensus.cc:374] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:24.260229 6046 raft_consensus.cc:729] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:24.260785 6046 consensus_queue.cc:260] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.263839 6046 ts_tablet_manager.cc:1428] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.042s user 0.007s sys 0.003s
I20260808 13:35:24.265566 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba39706cff65404485959865fff1772a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:24.270826 6033 ts_tablet_manager.cc:1428] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.099s user 0.005s sys 0.002s
I20260808 13:35:24.271050 6046 tablet_bootstrap.cc:492] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:24.260640 6913 leader_election.cc:288] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:24.288854 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 538d9a883e9d46668fde30876352d20e (DEFAULT_TABLE table=default.Table_114737be7a5c48089f667a5f02f76b5e [id=e3e121c36ff44ea0a3a98185c8cce381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.290297 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3682c9c656c4556bc13ae3aeddd7c5e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:24.291457 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9e8d48c55349058cf80bf7c2e149e7 (DEFAULT_TABLE table=default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.291738 5832 tablet_service.cc:1467] Processing CreateTablet for tablet 37495b29698940edb2924eb612e94bd7 (DEFAULT_TABLE table=default.Table_c3cd224a54354169a0cba0ad45c3257e [id=ed0521e8ecae4c609b2a7426e5a565c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.293895 5840 tablet_service.cc:1467] Processing CreateTablet for tablet d09eb729339e47eb8db88eb6509daab4 (DEFAULT_TABLE table=default.Table_03b5de95197845c1bc7ab06aec3d983d [id=af41016671a644dcaacf0b2602d83672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.294387 5834 tablet_service.cc:1467] Processing CreateTablet for tablet ba39706cff65404485959865fff1772a (DEFAULT_TABLE table=default.Table_fc7cb305e35541c781c3e3790cf8a0af [id=b885c5ab382c45809d6f896ebf8771bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.292634 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b9e8d48c55349058cf80bf7c2e149e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:24.296511 5832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37495b29698940edb2924eb612e94bd7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:24.298218 4835 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a8e18e51c2e24d3caa3d29e6645e46ae consensus state has no leader
I20260808 13:35:24.298218 5842 tablet_service.cc:1467] Processing CreateTablet for tablet b8e3245c8ba54f69bc329a5b75115127 (DEFAULT_TABLE table=default.Table_fd822a88bb39430eb487b2bdc43e86fb [id=093c441116e34908b83de704bbb58ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.298879 4835 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 265) took 2025 ms. Trace:
I20260808 13:35:24.297178 5835 tablet_service.cc:1467] Processing CreateTablet for tablet de6800eceb96475284fdbffb4d23e5e3 (DEFAULT_TABLE table=default.Table_9fa5c49027684fe6a85a46895e121932 [id=7fff2277bc2f4edd9aeccc15d58333c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.300642 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d09eb729339e47eb8db88eb6509daab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:24.301256 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6800eceb96475284fdbffb4d23e5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:24.299176 4835 rpcz_store.cc:276] 0808 13:35:22.273822 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.645502 (+1371680us) service_pool.cc:227] Handling call
0808 13:35:23.692924 (+ 47422us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:23.693598 (+ 674us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:23.694176 (+ 578us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:23.694460 (+ 284us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:24.297628 (+603168us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:24.298821 (+ 1193us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:24.145185 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:24.145932 (+ 747us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:24.145985 (+ 53us) write_op.cc:435] Acquired schema lock
0808 13:35:24.145997 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:24.146575 (+ 578us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:24.239669 (+ 93094us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:24.239770 (+ 101us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:24.240219 (+ 449us) tablet.cc:699] Row locks acquired
0808 13:35:24.240237 (+ 18us) write_op.cc:260] PREPARE: finished
0808 13:35:24.240458 (+ 221us) write_op.cc:270] Start()
0808 13:35:24.240611 (+ 153us) write_op.cc:276] Timestamp: P: 1786196124240395 usec, L: 0
0808 13:35:24.240630 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:24.241899 (+ 1269us) log.cc:842] Serialized 405 byte log entry
0808 13:35:24.250017 (+ 8118us) op_driver.cc:444] REPLICATION: finished
0808 13:35:24.279028 (+ 29011us) write_op.cc:301] APPLY: starting
0808 13:35:24.279100 (+ 72us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:24.279205 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:24.279318 (+ 113us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:24.279532 (+ 214us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:24.279690 (+ 158us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:24.279740 (+ 50us) write_op.cc:312] APPLY: finished
0808 13:35:24.279935 (+ 195us) log.cc:842] Serialized 23 byte log entry
0808 13:35:24.295773 (+ 15838us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.297189 (+ 1416us) write_op.cc:454] Released schema lock
0808 13:35:24.297274 (+ 85us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:24.297619 (+ 345us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.297671 (+ 52us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":28815,"apply.run_cpu_time_us":2226,"apply.run_wall_time_us":18890,"num_ops":1,"prepare.queue_time_us":449564,"prepare.run_cpu_time_us":4221,"prepare.run_wall_time_us":97131,"raft.queue_time_us":1174,"raft.run_cpu_time_us":764,"raft.run_wall_time_us":763,"replication_time_us":9273}]]}
I20260808 13:35:24.303758 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e3245c8ba54f69bc329a5b75115127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:24.304553 5843 tablet_service.cc:1467] Processing CreateTablet for tablet b1c41483119a442eb46816aa34f7f319 (DEFAULT_TABLE table=default.Table_a16be245c9864b969707b18ee86278d0 [id=8f151f8823464520ae72942a6c79d8bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.305307 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538d9a883e9d46668fde30876352d20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:24.305258 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d5bb15ded3d442699c050da99887a456"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:24.306668 6033 tablet_bootstrap.cc:492] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:24.307006 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5bb15ded3d442699c050da99887a456 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:24.307147 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:24.308069 5836 tablet_service.cc:1467] Processing CreateTablet for tablet fc3e9c6770fd402290b45eed85de6f6f (DEFAULT_TABLE table=default.Table_bf8907064282450eb869d5bb8ec9f1c1 [id=07969203fd174637a09f5b8e83bc7402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:24.308967 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba39706cff65404485959865fff1772a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:24.317380 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1c41483119a442eb46816aa34f7f319. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:24.323398 6953 consensus_meta.cc:287] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.743s user 0.011s sys 0.023s
I20260808 13:35:24.324000 6953 raft_consensus.cc:504] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.338948 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc3e9c6770fd402290b45eed85de6f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:24.353670 6033 tablet_bootstrap.cc:654] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:24.357460 5831 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c2b34c95244472beec43643e60ff5f with delete_type TABLET_DATA_DELETED (replaced by tablet b314734698ed423ba9cf4134db5846ec) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:24.359048 5880 raft_consensus.cc:2379] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:24.360181 6953 leader_election.cc:288] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:24.365674 5879 raft_consensus.cc:2454] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:24.369442 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet b809b185ee9e4c658a050c41c9abc4ad (table default.Table_1680269ff16246c382d6bccf44f8d52e [id=a15e38df2aee420fb9345cabd03dc842]) successfully deleted
I20260808 13:35:24.359732 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd8f12cda94c4eb9a848f002a4020b" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:24.387108 5642 raft_consensus.cc:3037] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:24.419658 5551 leader_election.cc:341] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:24.424324 5642 raft_consensus.cc:2454] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:24.431926 5772 leader_election.cc:302] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:35:24.448900 6026 ts_tablet_manager.cc:1460] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: Tablet startup took 615ms
I20260808 13:35:24.459719 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c2b34c95244472beec43643e60ff5f with delete_type TABLET_DATA_DELETED (replaced by tablet b314734698ed423ba9cf4134db5846ec) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:24.462100 4823 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c -> Peer 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135): Couldn't send request to peer 87e8b888041b4135af55792050defcb5. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260808 13:35:24.464778 7014 ts_tablet_manager.cc:1918] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:24.465197 7014 log.cc:1196] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/2cd2d0c9ab2c4704a74d06559bb913d6
I20260808 13:35:24.465947 7014 ts_tablet_manager.cc:1939] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:24.468871 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 2cd2d0c9ab2c4704a74d06559bb913d6 (table default.Table_34821e686c9b4e988ffdbb94388b2280 [id=ccd192d15e9d42b5ab0739dc606301e1]) successfully deleted
I20260808 13:35:24.483086 6585 tablet_replica.cc:330] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:24.384893 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c2b34c95244472beec43643e60ff5f with delete_type TABLET_DATA_DELETED (replaced by tablet b314734698ed423ba9cf4134db5846ec) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:24.499251 7014 tablet_replica.cc:330] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
W20260808 13:35:24.479321 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 264101us
I20260808 13:35:24.547647 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd8f12cda94c4eb9a848f002a4020b" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:24.559674 7013 ts_tablet_manager.cc:1905] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:24.564441 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 132587us
I20260808 13:35:24.572249 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 973a7afdcbf944aab8f11b3fd3995865
I20260808 13:35:24.573917 4842 catalog_manager.cc:6565] ReplaceTablet: tablet 973a7afdcbf944aab8f11b3fd3995865 deleted and replaced by tablet 3424ca753bcc4f448d314c56aba3323e
I20260808 13:35:24.574237 4842 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60270 (request call id 267) took 1732 ms. Trace:
I20260808 13:35:24.574498 4842 rpcz_store.cc:276] 0808 13:35:22.842366 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.148561 (+1306195us) service_pool.cc:227] Handling call
0808 13:35:24.574182 (+425621us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:24.451483 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:24.451919 (+ 436us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:24.451944 (+ 25us) write_op.cc:435] Acquired schema lock
0808 13:35:24.451965 (+ 21us) tablet.cc:660] Decoding operations
0808 13:35:24.452297 (+ 332us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:24.452340 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:24.452363 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:24.452577 (+ 214us) tablet.cc:699] Row locks acquired
0808 13:35:24.452589 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:24.452719 (+ 130us) write_op.cc:270] Start()
0808 13:35:24.452810 (+ 91us) write_op.cc:276] Timestamp: P: 1786196124452632 usec, L: 0
0808 13:35:24.452826 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:24.453313 (+ 487us) log.cc:842] Serialized 656 byte log entry
0808 13:35:24.561948 (+108635us) op_driver.cc:444] REPLICATION: finished
0808 13:35:24.562141 (+ 193us) write_op.cc:301] APPLY: starting
0808 13:35:24.562196 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:24.562327 (+ 131us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:24.562438 (+ 111us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:24.562857 (+ 419us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:24.563174 (+ 317us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:24.563242 (+ 68us) write_op.cc:312] APPLY: finished
0808 13:35:24.563422 (+ 180us) log.cc:842] Serialized 29 byte log entry
0808 13:35:24.563548 (+ 126us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.563612 (+ 64us) write_op.cc:454] Released schema lock
0808 13:35:24.563640 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:24.563801 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.563827 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":75,"apply.run_cpu_time_us":1841,"apply.run_wall_time_us":1841,"num_ops":2,"prepare.queue_time_us":297450,"prepare.run_cpu_time_us":2188,"prepare.run_wall_time_us":24734,"raft.queue_time_us":93,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":13,"replication_time_us":109049,"threads_started":1}]]}
W20260808 13:35:24.579645 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet e7616d81aa924933a9b24d44004725fa because the tablet was not found. No further retry: Not found: Tablet not found: e7616d81aa924933a9b24d44004725fa
I20260808 13:35:24.586328 6046 tablet_bootstrap.cc:654] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:24.588593 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 973a7afdcbf944aab8f11b3fd3995865 with delete_type TABLET_DATA_DELETED (replaced by tablet 3424ca753bcc4f448d314c56aba3323e) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:24.593106 5550 leader_election.cc:341] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:24.594170 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c083c8e19d4279af600670f5d1d78d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:24.596007 5550 leader_election.cc:341] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:24.597471 5550 leader_election.cc:302] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:24.597646 6585 raft_consensus.cc:2229] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:24.598757 6901 raft_consensus.cc:2726] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:24.599709 5550 leader_election.cc:302] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:24.600482 6901 raft_consensus.cc:2781] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:24.603111 6901 raft_consensus.cc:686] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:24.604292 6901 consensus_queue.cc:237] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:24.449349 6026 ts_tablet_manager.cc:1462] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: Trace:
0808 13:35:13.128488 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:13.136159 (+ 7671us) service_pool.cc:227] Handling call
0808 13:35:13.138097 (+ 1938us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:13.138211 (+ 114us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:13.365405 (+227194us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:13.546260 (+180855us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:13.765506 (+219246us) inbound_call.cc:173] Queueing success response
0808 13:35:23.309559 (+9544053us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0808 13:35:23.833220 (+523661us) ts_tablet_manager.cc:1429] Starting tablet replica
0808 13:35:23.833716 (+ 496us) tablet_replica.cc:259] Starting instrumentation
0808 13:35:23.834169 (+ 453us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":68209,"raft.queue_time_us":68380,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":177,"tablet-open.queue_time_us":9544191}
I20260808 13:35:24.615497 6026 tablet_bootstrap.cc:492] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:24.620936 5550 leader_election.cc:302] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:24.622177 7032 raft_consensus.cc:2726] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:24.624598 4841 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 261) took 2680 ms. Trace:
I20260808 13:35:24.625008 4841 rpcz_store.cc:276] 0808 13:35:21.944080 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.621942 (+1677862us) service_pool.cc:227] Handling call
0808 13:35:23.722169 (+100227us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:23.723838 (+ 1669us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:24.623780 (+899942us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:24.624180 (+ 400us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:24.624518 (+ 338us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:24.242757 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:24.243607 (+ 850us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:24.382106 (+138499us) write_op.cc:435] Acquired schema lock
0808 13:35:24.382129 (+ 23us) tablet.cc:660] Decoding operations
0808 13:35:24.382875 (+ 746us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:24.382934 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:24.382988 (+ 54us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:24.436068 (+ 53080us) tablet.cc:699] Row locks acquired
0808 13:35:24.436083 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:24.436213 (+ 130us) write_op.cc:270] Start()
0808 13:35:24.436310 (+ 97us) write_op.cc:276] Timestamp: P: 1786196124436142 usec, L: 0
0808 13:35:24.436328 (+ 18us) op_driver.cc:331] REPLICATION: starting
0808 13:35:24.436895 (+ 567us) log.cc:842] Serialized 433 byte log entry
0808 13:35:24.451405 (+ 14510us) op_driver.cc:444] REPLICATION: finished
0808 13:35:24.473851 (+ 22446us) write_op.cc:301] APPLY: starting
0808 13:35:24.473939 (+ 88us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:24.474102 (+ 163us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:24.474271 (+ 169us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:24.474799 (+ 528us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:24.474936 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:24.474983 (+ 47us) write_op.cc:312] APPLY: finished
0808 13:35:24.475209 (+ 226us) log.cc:842] Serialized 29 byte log entry
0808 13:35:24.623307 (+148098us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.623400 (+ 93us) write_op.cc:454] Released schema lock
0808 13:35:24.623463 (+ 63us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:24.623781 (+ 318us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.623834 (+ 53us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":22219,"apply.run_cpu_time_us":2523,"apply.run_wall_time_us":150247,"num_ops":2,"prepare.queue_time_us":517357,"prepare.run_cpu_time_us":3598,"prepare.run_wall_time_us":194651,"raft.queue_time_us":1600,"raft.run_cpu_time_us":604,"raft.run_wall_time_us":3959,"replication_time_us":15007}]]}
I20260808 13:35:24.382349 6953 raft_consensus.cc:482] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:24.632510 7014 raft_consensus.cc:2229] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:24.633383 6585 raft_consensus.cc:2258] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:24.633554 7014 raft_consensus.cc:2258] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:24.636987 7014 ts_tablet_manager.cc:1905] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:24.523631 7049 raft_consensus.cc:482] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:24.661401 7049 raft_consensus.cc:504] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.664885 7049 leader_election.cc:288] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:24.646086 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 76118.5us
I20260808 13:35:24.677088 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c083c8e19d4279af600670f5d1d78d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:24.677906 5637 raft_consensus.cc:3037] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:24.678385 5283 leader_election.cc:341] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:24.680696 5283 leader_election.cc:341] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:24.684809 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f1618a1d044898bffbc82107e90840" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:24.685352 5642 raft_consensus.cc:2454] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:24.687991 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet 973a7afdcbf944aab8f11b3fd3995865 with delete_type TABLET_DATA_DELETED (replaced by tablet 3424ca753bcc4f448d314c56aba3323e) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:24.689496 7047 consensus_queue.cc:1035] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:24.694485 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a8e18e51c2e24d3caa3d29e6645e46ae (table default.Table_321b4eb3dd024925af1cb354d3fe1567 [id=3c108c9e89424d9a9e11554601cf427c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a8e18e51c2e24d3caa3d29e6645e46ae consensus state has no leader
I20260808 13:35:24.689364 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3682c9c656c4556bc13ae3aeddd7c5e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:24.726778 5551 leader_election.cc:341] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:24.727317 5551 leader_election.cc:302] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:24.728667 7032 raft_consensus.cc:2726] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:24.736004 7014 ts_tablet_manager.cc:1918] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:24.737514 7014 log.cc:1196] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/2173e97590be43f9a24f55689d796269
W20260808 13:35:24.724793 5285 consensus_peers.cc:487] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2cd2d0c9ab2c4704a74d06559bb913d6. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:24.739872 5285 leader_election.cc:302] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:24.739086 7014 ts_tablet_manager.cc:1939] T 2173e97590be43f9a24f55689d796269 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:24.749686 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 196) took 1133 ms. Trace:
I20260808 13:35:24.752352 7014 rpcz_store.cc:276] 0808 13:35:23.616696 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.626710 (+ 10014us) service_pool.cc:227] Handling call
0808 13:35:24.498950 (+872240us) ts_tablet_manager.cc:1145] Deleting tablet 2173e97590be43f9a24f55689d796269
0808 13:35:24.499036 (+ 86us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:24.670573 (+171537us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.734180 (+ 63607us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.749607 (+ 15427us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":132119,"tablet-delete.queue_time_us":871623}
I20260808 13:35:24.758378 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 2173e97590be43f9a24f55689d796269 (table default.Table_d052232acdff41f394004ddc3252bbbd [id=6017dfb7d1574e88a452e2d539b1aa03]) successfully deleted
I20260808 13:35:24.626037 6810 raft_consensus.cc:2781] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:24.760519 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 896389941179456d9d0ed3cd14726f11
I20260808 13:35:24.761962 4840 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 268) took 1027 ms. Trace:
I20260808 13:35:24.760705 7014 tablet_replica.cc:330] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:24.764092 7014 raft_consensus.cc:2229] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:24.766666 7014 raft_consensus.cc:2258] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:24.769763 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 896389941179456d9d0ed3cd14726f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:24.770818 6026 tablet_bootstrap.cc:654] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:24.762249 4840 rpcz_store.cc:276] 0808 13:35:23.734818 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.162928 (+428110us) service_pool.cc:227] Handling call
0808 13:35:24.165001 (+ 2073us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:24.166362 (+ 1361us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:24.166601 (+ 239us) catalog_manager.cc:2761] Locking tablets
0808 13:35:24.167257 (+ 656us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:24.760079 (+592822us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:24.760306 (+ 227us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:24.760434 (+ 128us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:24.761902 (+ 1468us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:24.476360 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:24.476814 (+ 454us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:24.476841 (+ 27us) write_op.cc:435] Acquired schema lock
0808 13:35:24.476857 (+ 16us) tablet.cc:660] Decoding operations
0808 13:35:24.477205 (+ 348us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:24.477240 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:24.477270 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:24.483449 (+ 6179us) tablet.cc:699] Row locks acquired
0808 13:35:24.483468 (+ 19us) write_op.cc:260] PREPARE: finished
0808 13:35:24.484674 (+ 1206us) write_op.cc:270] Start()
0808 13:35:24.484885 (+ 211us) write_op.cc:276] Timestamp: P: 1786196124483517 usec, L: 0
0808 13:35:24.484920 (+ 35us) op_driver.cc:331] REPLICATION: starting
0808 13:35:24.485457 (+ 537us) log.cc:842] Serialized 756 byte log entry
0808 13:35:24.754984 (+269527us) op_driver.cc:444] REPLICATION: finished
0808 13:35:24.755357 (+ 373us) write_op.cc:301] APPLY: starting
0808 13:35:24.755475 (+ 118us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:24.755711 (+ 236us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:24.755974 (+ 263us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:24.756385 (+ 411us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:24.756724 (+ 339us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:24.756832 (+ 108us) write_op.cc:312] APPLY: finished
0808 13:35:24.757108 (+ 276us) log.cc:842] Serialized 29 byte log entry
0808 13:35:24.758058 (+ 950us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.758138 (+ 80us) write_op.cc:454] Released schema lock
0808 13:35:24.758199 (+ 61us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":103,"num_ops":2,"prepare.queue_time_us":306878,"prepare.run_cpu_time_us":2431,"prepare.run_wall_time_us":12674,"raft.queue_time_us":3231,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":12,"replication_time_us":269989}]]}
W20260808 13:35:24.774348 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 104037us
I20260808 13:35:24.786054 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_feb9646b81ef46c7aa00ea076ae7240f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:24.787591 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_feb9646b81ef46c7aa00ea076ae7240f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:24.738787 6796 consensus_queue.cc:1035] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:24.763041 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet 896389941179456d9d0ed3cd14726f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:24.773583 5345 consensus_meta.cc:287] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c: Time spent flushing consensus metadata: real 0.770s user 0.008s sys 0.000s
I20260808 13:35:24.795925 5345 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 214) took 1069 ms. Trace:
I20260808 13:35:24.796217 5345 rpcz_store.cc:276] 0808 13:35:23.726984 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.740574 (+ 13590us) service_pool.cc:227] Handling call
0808 13:35:23.742190 (+ 1616us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:23.742379 (+ 189us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:23.865290 (+122911us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.795449 (+930159us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:24.795854 (+ 405us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260808 13:35:24.773988 5336 consensus_meta.cc:287] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c: Time spent flushing consensus metadata: real 0.517s user 0.007s sys 0.000s
W20260808 13:35:24.774070 5335 consensus_meta.cc:287] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c: Time spent flushing consensus metadata: real 0.972s user 0.007s sys 0.007s
I20260808 13:35:24.798029 5335 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 217) took 1018 ms. Trace:
I20260808 13:35:24.798964 5351 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 212) took 1125 ms. Trace:
I20260808 13:35:24.799228 5336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 215) took 1034 ms. Trace:
I20260808 13:35:24.799387 5351 rpcz_store.cc:276] 0808 13:35:23.674055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.674235 (+ 180us) service_pool.cc:227] Handling call
0808 13:35:23.676008 (+ 1773us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:23.676126 (+ 118us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:24.523531 (+847405us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.795449 (+271918us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:24.798909 (+ 3460us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2999}
I20260808 13:35:24.798357 5335 rpcz_store.cc:276] 0808 13:35:23.779207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.779394 (+ 187us) service_pool.cc:227] Handling call
0808 13:35:23.780799 (+ 1405us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:23.780885 (+ 86us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:23.798233 (+ 17348us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.797561 (+999328us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:24.797978 (+ 417us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":24}
W20260808 13:35:24.800990 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 61240us
I20260808 13:35:24.784405 6046 tablet_bootstrap.cc:492] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:24.802969 6046 ts_tablet_manager.cc:1397] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.532s user 0.010s sys 0.006s
I20260808 13:35:24.801537 6796 raft_consensus.cc:2781] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:24.799597 5336 rpcz_store.cc:276] 0808 13:35:23.764434 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.764608 (+ 174us) service_pool.cc:227] Handling call
0808 13:35:23.766084 (+ 1476us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:23.766170 (+ 86us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:24.097720 (+331550us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.797561 (+699841us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:24.799177 (+ 1616us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":948}
I20260808 13:35:24.803562 6796 raft_consensus.cc:482] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:24.804421 6796 raft_consensus.cc:3037] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:24.630770 5833 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 193) took 1114 ms. Trace:
I20260808 13:35:24.775686 7049 raft_consensus.cc:482] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:24.807216 5334 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 218) took 1019 ms. Trace:
I20260808 13:35:24.807415 7049 raft_consensus.cc:504] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.826414 6026 tablet_bootstrap.cc:492] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:24.826926 6026 ts_tablet_manager.cc:1397] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.212s user 0.004s sys 0.007s
I20260808 13:35:24.833285 6026 raft_consensus.cc:348] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.834120 6026 raft_consensus.cc:374] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:24.834465 6026 raft_consensus.cc:729] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:24.835073 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_ca3570782c0f48a88952f5f442fdbde6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:24.835242 6026 consensus_queue.cc:260] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:24.836557 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca3570782c0f48a88952f5f442fdbde6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:24.839299 6033 tablet_bootstrap.cc:492] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:24.839367 6585 ts_tablet_manager.cc:1905] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:24.839743 6033 ts_tablet_manager.cc:1397] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.533s user 0.010s sys 0.021s
I20260808 13:35:24.840587 6026 ts_tablet_manager.cc:1428] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.013s user 0.006s sys 0.001s
I20260808 13:35:24.852541 5637 raft_consensus.cc:2454] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:24.807771 5334 rpcz_store.cc:276] 0808 13:35:23.787444 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.798797 (+ 11353us) service_pool.cc:227] Handling call
0808 13:35:23.810765 (+ 11968us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:23.810852 (+ 87us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:24.553194 (+742342us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.806095 (+252901us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:24.807158 (+ 1063us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4418}
I20260808 13:35:24.815546 5355 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 216) took 1036 ms. Trace:
I20260808 13:35:24.810868 6046 raft_consensus.cc:348] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.819180 7041 consensus_queue.cc:1035] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:24.857213 6046 raft_consensus.cc:374] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:24.856963 5355 rpcz_store.cc:276] 0808 13:35:23.778923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.808964 (+ 30041us) service_pool.cc:227] Handling call
0808 13:35:23.826799 (+ 17835us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:23.826932 (+ 133us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:24.751050 (+924118us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.815108 (+ 64058us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:24.815493 (+ 385us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:24.857694 6046 raft_consensus.cc:729] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:24.823940 7039 consensus_queue.cc:1035] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:24.858330 6046 consensus_queue.cc:260] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.860700 6046 ts_tablet_manager.cc:1428] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.057s user 0.007s sys 0.000s
I20260808 13:35:24.861618 6046 tablet_bootstrap.cc:492] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:24.638831 6953 raft_consensus.cc:504] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.900283 6585 ts_tablet_manager.cc:1918] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:24.900663 6585 log.cc:1196] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/543debff6da54d26bf62260bf8721ff7
I20260808 13:35:24.901556 6585 ts_tablet_manager.cc:1939] T 543debff6da54d26bf62260bf8721ff7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:24.902318 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 196) took 1377 ms. Trace:
I20260808 13:35:24.902693 6585 rpcz_store.cc:276] 0808 13:35:23.525227 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.526000 (+ 773us) service_pool.cc:227] Handling call
0808 13:35:24.482926 (+956926us) ts_tablet_manager.cc:1145] Deleting tablet 543debff6da54d26bf62260bf8721ff7
0808 13:35:24.482969 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:24.884690 (+401721us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.900046 (+ 15356us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.902264 (+ 2218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":956373}
I20260808 13:35:24.908605 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_de3d9a6e588d44f698c656a21558e2ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:24.910377 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_03b5de95197845c1bc7ab06aec3d983d" } new_table_name: "default.Table_233ac5cc38504a5eb9b9cbf426cc19c7" modify_external_catalogs: true
I20260808 13:35:24.865295 7049 leader_election.cc:288] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:24.923086 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de3d9a6e588d44f698c656a21558e2ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:24.935323 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 197) took 1408 ms. Trace:
I20260808 13:35:24.935823 6585 rpcz_store.cc:276] 0808 13:35:23.526935 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.527058 (+ 123us) service_pool.cc:227] Handling call
0808 13:35:24.934729 (+1407671us) ts_tablet_manager.cc:1145] Deleting tablet e2a95712de254c13b3b6dc4fe5cb15f7
0808 13:35:24.934795 (+ 66us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:24.935219 (+ 424us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1407109}
I20260808 13:35:24.921146 5332 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 220) took 1106 ms. Trace:
I20260808 13:35:24.958748 5332 rpcz_store.cc:276] 0808 13:35:23.815049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.823531 (+ 8482us) service_pool.cc:227] Handling call
0808 13:35:23.830674 (+ 7143us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:23.830751 (+ 77us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:24.537002 (+706251us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.879433 (+342431us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:24.921090 (+ 41657us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5544}
I20260808 13:35:24.920495 6046 tablet_bootstrap.cc:654] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:24.922731 5354 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46328 (request call id 219) took 1115 ms. Trace:
I20260808 13:35:24.959952 5354 rpcz_store.cc:276] 0808 13:35:23.807525 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.811227 (+ 3702us) service_pool.cc:227] Handling call
0808 13:35:23.812396 (+ 1169us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:23.812465 (+ 69us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:24.612673 (+800208us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.916203 (+303530us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:24.922675 (+ 6472us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:24.937006 5285 leader_election.cc:302] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:24.963824 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 198) took 1435 ms. Trace:
I20260808 13:35:24.964183 6585 rpcz_store.cc:276] 0808 13:35:23.528801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.529075 (+ 274us) service_pool.cc:227] Handling call
0808 13:35:24.936932 (+1407857us) ts_tablet_manager.cc:1145] Deleting tablet cfc9a6ec9a86431fb46e50fcaa658d33
0808 13:35:24.936989 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:24.963756 (+ 26767us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1407290}
I20260808 13:35:24.945802 7013 ts_tablet_manager.cc:1918] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:24.965682 7013 log.cc:1196] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/b809b185ee9e4c658a050c41c9abc4ad
I20260808 13:35:24.919739 6796 raft_consensus.cc:504] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.807046 5833 rpcz_store.cc:276] 0808 13:35:23.516479 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.518762 (+ 2283us) service_pool.cc:227] Handling call
0808 13:35:23.520120 (+ 1358us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:23.520214 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:24.315009 (+794795us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.368437 (+ 53428us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:24.626163 (+257726us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:24.977607 6796 leader_election.cc:288] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:24.981175 6033 raft_consensus.cc:348] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.981974 6033 raft_consensus.cc:374] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:24.982264 6033 raft_consensus.cc:729] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:24.984328 6033 consensus_queue.cc:260] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.993930 6046 tablet_bootstrap.cc:492] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:24.995182 6046 ts_tablet_manager.cc:1397] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.134s user 0.010s sys 0.013s
I20260808 13:35:24.995491 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f1618a1d044898bffbc82107e90840" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:24.995781 7013 ts_tablet_manager.cc:1939] T b809b185ee9e4c658a050c41c9abc4ad P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:24.996150 5879 raft_consensus.cc:2454] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:24.996691 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 198) took 1448 ms. Trace:
I20260808 13:35:24.996929 5833 tablet_service.cc:1514] Processing DeleteTablet for tablet 973a7afdcbf944aab8f11b3fd3995865 with delete_type TABLET_DATA_DELETED (replaced by tablet 3424ca753bcc4f448d314c56aba3323e) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:24.997020 7013 rpcz_store.cc:276] 0808 13:35:23.547837 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.548074 (+ 237us) service_pool.cc:227] Handling call
0808 13:35:23.816620 (+268546us) ts_tablet_manager.cc:1145] Deleting tablet b809b185ee9e4c658a050c41c9abc4ad
0808 13:35:23.816703 (+ 83us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:24.777623 (+960920us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.945370 (+167747us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:24.996621 (+ 51251us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4299,"tablet-delete.queue_time_us":267572,"thread_start_us":317779,"threads_started":1}
I20260808 13:35:24.997586 6046 raft_consensus.cc:348] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:24.998301 6046 raft_consensus.cc:374] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:24.998665 6046 raft_consensus.cc:729] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:24.998687 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet b809b185ee9e4c658a050c41c9abc4ad (table default.Table_1680269ff16246c382d6bccf44f8d52e [id=a15e38df2aee420fb9345cabd03dc842]) successfully deleted
I20260808 13:35:24.987064 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 199) took 1457 ms. Trace:
I20260808 13:35:24.999341 6585 rpcz_store.cc:276] 0808 13:35:23.530081 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.530186 (+ 105us) service_pool.cc:227] Handling call
0808 13:35:24.965079 (+1434893us) ts_tablet_manager.cc:1145] Deleting tablet 2fa77744bcfb4a5cace730fd9f9fd1ca
0808 13:35:24.965132 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:24.986992 (+ 21860us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1434290}
I20260808 13:35:24.999293 6046 consensus_queue.cc:260] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.000535 7039 raft_consensus.cc:2781] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:25.000945 7039 raft_consensus.cc:686] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:25.001154 6046 ts_tablet_manager.cc:1428] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260808 13:35:25.002099 6046 tablet_bootstrap.cc:492] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:25.001677 7039 consensus_queue.cc:237] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.004782 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 543debff6da54d26bf62260bf8721ff7 (table default.Table_062151f3d1444efc8a42ba6cd06e19c9 [id=68dee345bd2f4fc1823048b3da8d944f]) successfully deleted
W20260808 13:35:25.005482 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet e2a95712de254c13b3b6dc4fe5cb15f7 because the tablet was not found. No further retry: Not found: Tablet not found: e2a95712de254c13b3b6dc4fe5cb15f7
W20260808 13:35:25.006054 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet cfc9a6ec9a86431fb46e50fcaa658d33 because the tablet was not found. No further retry: Not found: Tablet not found: cfc9a6ec9a86431fb46e50fcaa658d33
I20260808 13:35:25.007936 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 200) took 1250 ms. Trace:
I20260808 13:35:25.008224 6585 rpcz_store.cc:276] 0808 13:35:23.757426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.757575 (+ 149us) service_pool.cc:227] Handling call
0808 13:35:25.000243 (+1242668us) ts_tablet_manager.cc:1145] Deleting tablet a0b29786cb404bc2b01473c3b8390a50
0808 13:35:25.000283 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.007875 (+ 7592us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1242133}
I20260808 13:35:25.012169 6953 leader_election.cc:288] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:25.013931 5833 tablet_service.cc:1514] Processing DeleteTablet for tablet 896389941179456d9d0ed3cd14726f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:25.015690 6046 tablet_bootstrap.cc:654] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:25.018587 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 2fa77744bcfb4a5cace730fd9f9fd1ca because the tablet was not found. No further retry: Not found: Tablet not found: 2fa77744bcfb4a5cace730fd9f9fd1ca
W20260808 13:35:25.019253 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet a0b29786cb404bc2b01473c3b8390a50 because the tablet was not found. No further retry: Not found: Tablet not found: a0b29786cb404bc2b01473c3b8390a50
W20260808 13:35:25.020010 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 5b6e6ce417cc45fd90f5858c30d37a47 because the tablet was not found. No further retry: Not found: Tablet not found: 5b6e6ce417cc45fd90f5858c30d37a47
I20260808 13:35:25.023243 6585 tablet_replica.cc:330] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:25.023391 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f1618a1d044898bffbc82107e90840" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:25.024260 5881 raft_consensus.cc:3037] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:25.027324 7041 consensus_queue.cc:1035] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:25.029188 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e18e51c2e24d3caa3d29e6645e46ae" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:25.040407 6585 raft_consensus.cc:2229] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:25.040931 6585 raft_consensus.cc:2258] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:25.042181 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 81fb3e099f7f4544b3c100070f0d14ef because the tablet was not found. No further retry: Not found: Tablet not found: 81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:25.024184 7013 tablet_replica.cc:330] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:25.044044 7013 raft_consensus.cc:2229] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:25.044281 6033 ts_tablet_manager.cc:1428] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.204s user 0.005s sys 0.003s
I20260808 13:35:25.044554 7013 raft_consensus.cc:2258] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:25.045984 5283 consensus_peers.cc:487] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:25.047252 5283 consensus_peers.cc:487] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:25.048342 5283 leader_election.cc:341] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:25.052382 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d09eb729339e47eb8db88eb6509daab4 consensus state has no leader
W20260808 13:35:25.053908 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 278910us
I20260808 13:35:25.054069 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b75082dac94348fab668160895573637" } modify_external_catalogs: true
I20260808 13:35:25.057147 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b75082dac94348fab668160895573637" } modify_external_catalogs: true
W20260808 13:35:25.058759 5771 leader_election.cc:341] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:25.046540 7013 ts_tablet_manager.cc:1905] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:25.066431 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_cab0814763554410aa969d5b417710fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:25.074477 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cab0814763554410aa969d5b417710fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:25.076990 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea8f5322a404837b91e1d96749926b7 (DEFAULT_TABLE table=default.Table_c0f49fa8313743fbb5ea5d76f5262610 [id=eff3697da0d14a919bdb6b1fa4536f35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:25.077977 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea8f5322a404837b91e1d96749926b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:25.093281 6046 tablet_bootstrap.cc:492] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:25.093808 6046 ts_tablet_manager.cc:1397] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.092s user 0.007s sys 0.003s
I20260808 13:35:25.100525 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e7f2daaad4f48a79b94344ff35e98" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:25.116782 6046 raft_consensus.cc:348] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.117473 6046 raft_consensus.cc:374] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:25.117697 6046 raft_consensus.cc:729] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:25.118393 6046 consensus_queue.cc:260] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.120740 6046 ts_tablet_manager.cc:1428] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.027s user 0.004s sys 0.004s
I20260808 13:35:25.122243 5881 raft_consensus.cc:2454] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:25.123234 6585 ts_tablet_manager.cc:1905] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:25.132939 5283 leader_election.cc:302] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:25.134841 7039 raft_consensus.cc:2781] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:25.135295 6046 tablet_bootstrap.cc:492] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:25.135713 7039 raft_consensus.cc:686] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:25.138217 7039 consensus_queue.cc:237] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.144137 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 356e15bbcd26409a9d6302599816e15f (DEFAULT_TABLE table=default.Table_322ce0f117834a81978e588b16196c2a [id=9651f57557304d8eb476787530d19f39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:25.144573 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 62d68ab73bae4dd9af46e907b1fd198c (DEFAULT_TABLE table=default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:25.145830 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356e15bbcd26409a9d6302599816e15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:25.146281 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 62d68ab73bae4dd9af46e907b1fd198c (DEFAULT_TABLE table=default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:25.148445 6033 tablet_bootstrap.cc:492] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:25.148473 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 62d68ab73bae4dd9af46e907b1fd198c (DEFAULT_TABLE table=default.Table_4159935d23b8404ea7b50d91562d2fc7 [id=46d67ef595cf4e9d9516c7775cc56ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:25.053076 6810 raft_consensus.cc:686] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
W20260808 13:35:25.150460 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 96683.8us
I20260808 13:35:25.151600 6810 consensus_queue.cc:237] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.163019 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62d68ab73bae4dd9af46e907b1fd198c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:25.164757 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62d68ab73bae4dd9af46e907b1fd198c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:25.168958 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e18e51c2e24d3caa3d29e6645e46ae" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:25.169917 5638 raft_consensus.cc:2454] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:25.181291 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea8f5322a404837b91e1d96749926b7 (DEFAULT_TABLE table=default.Table_c0f49fa8313743fbb5ea5d76f5262610 [id=eff3697da0d14a919bdb6b1fa4536f35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:25.184127 6046 tablet_bootstrap.cc:654] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:25.195225 5834 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea8f5322a404837b91e1d96749926b7 (DEFAULT_TABLE table=default.Table_c0f49fa8313743fbb5ea5d76f5262610 [id=eff3697da0d14a919bdb6b1fa4536f35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:25.197206 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea8f5322a404837b91e1d96749926b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:25.197227 5285 leader_election.cc:302] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:25.200955 5600 tablet_service.cc:1467] Processing CreateTablet for tablet 356e15bbcd26409a9d6302599816e15f (DEFAULT_TABLE table=default.Table_322ce0f117834a81978e588b16196c2a [id=9651f57557304d8eb476787530d19f39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:25.202067 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e7f2daaad4f48a79b94344ff35e98" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:25.203251 5630 raft_consensus.cc:2454] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:25.204941 7014 ts_tablet_manager.cc:1905] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:25.207329 6953 raft_consensus.cc:482] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:25.208019 6953 raft_consensus.cc:504] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:25.211231 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 105944us
I20260808 13:35:25.160256 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 356e15bbcd26409a9d6302599816e15f (DEFAULT_TABLE table=default.Table_322ce0f117834a81978e588b16196c2a [id=9651f57557304d8eb476787530d19f39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:25.216845 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356e15bbcd26409a9d6302599816e15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:25.175052 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f1618a1d044898bffbc82107e90840" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:25.219808 5636 raft_consensus.cc:3037] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:25.220186 7022 consensus_queue.cc:1035] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:25.223423 5772 leader_election.cc:302] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:25.223878 6953 leader_election.cc:288] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:25.224537 6953 raft_consensus.cc:2781] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:25.225014 6953 raft_consensus.cc:482] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:25.227156 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62d68ab73bae4dd9af46e907b1fd198c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:25.227592 6026 tablet_bootstrap.cc:492] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:25.231886 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c083c8e19d4279af600670f5d1d78d" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:25.233439 5629 raft_consensus.cc:2379] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:25.240692 6913 raft_consensus.cc:2781] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:25.241358 6913 raft_consensus.cc:482] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:25.241919 6913 raft_consensus.cc:3037] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:25.244728 6033 tablet_bootstrap.cc:654] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:25.244663 5771 leader_election.cc:341] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:25.248399 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c083c8e19d4279af600670f5d1d78d" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:25.225536 6953 raft_consensus.cc:3037] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:25.262223 5771 leader_election.cc:302] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:25.269841 6585 ts_tablet_manager.cc:1918] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:25.271821 7022 raft_consensus.cc:3037] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:25.291085 5772 consensus_peers.cc:487] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:25.292032 6026 tablet_bootstrap.cc:654] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:25.293416 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea8f5322a404837b91e1d96749926b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:25.296098 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356e15bbcd26409a9d6302599816e15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:25.308881 7022 raft_consensus.cc:2726] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:25.309863 6585 log.cc:1196] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/2173e97590be43f9a24f55689d796269
I20260808 13:35:25.321281 6585 ts_tablet_manager.cc:1939] T 2173e97590be43f9a24f55689d796269 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:25.322458 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 204) took 1219 ms. Trace:
I20260808 13:35:25.322865 6585 rpcz_store.cc:276] 0808 13:35:24.103451 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.103557 (+ 106us) service_pool.cc:227] Handling call
0808 13:35:25.021150 (+917593us) ts_tablet_manager.cc:1145] Deleting tablet 2173e97590be43f9a24f55689d796269
0808 13:35:25.021189 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.165389 (+144200us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:25.269599 (+104210us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:25.322406 (+ 52807us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":916460}
I20260808 13:35:25.325935 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 205) took 1222 ms. Trace:
I20260808 13:35:25.330881 6585 rpcz_store.cc:276] 0808 13:35:24.103842 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.104162 (+ 320us) service_pool.cc:227] Handling call
0808 13:35:25.325654 (+1221492us) ts_tablet_manager.cc:1145] Deleting tablet 6d8018bf8ff843d8818fbba9473acf47
0808 13:35:25.325692 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.325886 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1220601}
I20260808 13:35:25.318813 6033 tablet_bootstrap.cc:492] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:25.334012 6033 ts_tablet_manager.cc:1397] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.186s user 0.011s sys 0.006s
I20260808 13:35:25.337666 6033 raft_consensus.cc:348] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.353296 6033 raft_consensus.cc:374] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:25.339440 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 207) took 1235 ms. Trace:
I20260808 13:35:25.354648 6585 rpcz_store.cc:276] 0808 13:35:24.104544 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.104991 (+ 447us) service_pool.cc:227] Handling call
0808 13:35:25.339103 (+1234112us) ts_tablet_manager.cc:1145] Deleting tablet 196d80d806b0432cacf2ca6aa0da8c6b
0808 13:35:25.339144 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.339392 (+ 248us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1220091}
I20260808 13:35:25.354359 6033 raft_consensus.cc:729] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:25.358500 6033 consensus_queue.cc:260] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.325122 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 2173e97590be43f9a24f55689d796269 (table default.Table_d052232acdff41f394004ddc3252bbbd [id=6017dfb7d1574e88a452e2d539b1aa03]) successfully deleted
W20260808 13:35:25.363634 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 6d8018bf8ff843d8818fbba9473acf47 because the tablet was not found. No further retry: Not found: Tablet not found: 6d8018bf8ff843d8818fbba9473acf47
I20260808 13:35:25.154568 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
W20260808 13:35:25.367717 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 196d80d806b0432cacf2ca6aa0da8c6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 196d80d806b0432cacf2ca6aa0da8c6b already in progress: creating tablet
I20260808 13:35:25.368700 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 203) took 1265 ms. Trace:
I20260808 13:35:25.369145 6585 rpcz_store.cc:276] 0808 13:35:24.103082 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.105454 (+ 2372us) service_pool.cc:227] Handling call
0808 13:35:25.355959 (+1250505us) ts_tablet_manager.cc:1145] Deleting tablet d2ac4c366363468aa62b73e0f1e04532
0808 13:35:25.356014 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.368643 (+ 12629us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1225157}
I20260808 13:35:25.370842 6585 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 218) took 1080 ms. Trace:
I20260808 13:35:25.371199 6585 rpcz_store.cc:276] 0808 13:35:24.321871 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.350693 (+ 28822us) service_pool.cc:227] Handling call
0808 13:35:25.370506 (+1019813us) ts_tablet_manager.cc:1145] Deleting tablet 86c2b34c95244472beec43643e60ff5f
0808 13:35:25.370549 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.370793 (+ 244us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1012275}
I20260808 13:35:25.372362 6585 tablet_replica.cc:330] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:25.328002 7047 raft_consensus.cc:482] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:25.376889 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet d2ac4c366363468aa62b73e0f1e04532 because the tablet was not found. No further retry: Not found: Tablet not found: d2ac4c366363468aa62b73e0f1e04532
I20260808 13:35:25.375200 7047 raft_consensus.cc:504] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.378779 6585 raft_consensus.cc:2229] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:25.379369 6585 raft_consensus.cc:2258] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:25.335994 6953 raft_consensus.cc:504] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.383181 6953 leader_election.cc:288] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:25.383872 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 86c2b34c95244472beec43643e60ff5f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 86c2b34c95244472beec43643e60ff5f already in progress: creating tablet
I20260808 13:35:25.382496 6585 ts_tablet_manager.cc:1905] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:25.386101 6033 ts_tablet_manager.cc:1428] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.052s user 0.006s sys 0.001s
I20260808 13:35:25.384404 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e7f2daaad4f48a79b94344ff35e98" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:25.387392 5644 raft_consensus.cc:3037] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:25.378585 6026 tablet_bootstrap.cc:492] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:25.395249 6026 ts_tablet_manager.cc:1397] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.168s user 0.020s sys 0.024s
I20260808 13:35:25.399006 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e7f2daaad4f48a79b94344ff35e98" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:25.399188 6026 raft_consensus.cc:348] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.400034 6026 raft_consensus.cc:374] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:25.399919 5384 raft_consensus.cc:3037] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:25.400282 6026 raft_consensus.cc:729] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:25.402258 5644 raft_consensus.cc:2454] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:25.366578 5636 raft_consensus.cc:2454] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:25.406495 5772 leader_election.cc:302] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:25.407756 7032 raft_consensus.cc:482] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:25.404857 6026 consensus_queue.cc:260] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.408316 7032 raft_consensus.cc:504] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.412227 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a541ec35b1a44109866fabc953f14cc6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:25.413911 5551 leader_election.cc:341] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:25.419570 6913 raft_consensus.cc:504] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.431555 7032 leader_election.cc:288] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:25.432874 6026 ts_tablet_manager.cc:1428] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.037s user 0.004s sys 0.003s
I20260808 13:35:25.433838 6026 tablet_bootstrap.cc:492] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:25.448267 5642 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s user 0.013s sys 0.006s
W20260808 13:35:25.448635 5642 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s user 0.013s sys 0.006s
I20260808 13:35:25.454849 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e7f2daaad4f48a79b94344ff35e98" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:25.456461 5644 raft_consensus.cc:2379] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:25.448096 4840 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:25.461055 4840 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:25.479892 6033 tablet_bootstrap.cc:492] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:25.451494 6953 raft_consensus.cc:2781] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:25.423404 7047 leader_election.cc:288] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:25.427075 6796 raft_consensus.cc:482] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:25.486250 6953 raft_consensus.cc:686] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:25.486900 6796 raft_consensus.cc:504] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.487691 6953 consensus_queue.cc:237] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.468250 7013 ts_tablet_manager.cc:1918] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:25.494840 7032 consensus_queue.cc:1035] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:25.487766 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e7f2daaad4f48a79b94344ff35e98" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:25.502599 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a541ec35b1a44109866fabc953f14cc6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:25.495558 7013 log.cc:1196] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/a0b29786cb404bc2b01473c3b8390a50
W20260808 13:35:25.506990 5550 leader_election.cc:341] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:25.507408 5550 leader_election.cc:302] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:25.507717 7013 ts_tablet_manager.cc:1939] T a0b29786cb404bc2b01473c3b8390a50 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:25.511049 5283 leader_election.cc:302] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:25.524988 4841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 2cd6945ecccc4684a3fa5b2a1f07057f
I20260808 13:35:25.532773 5609 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:25 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:25.543018 7032 raft_consensus.cc:2726] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:25.560169 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_8e189703ccf14f70903828148ce964df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:25.562083 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e189703ccf14f70903828148ce964df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:25.582079 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:25 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:25.461938 4840 catalog_manager.cc:5495] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:25.585166 4840 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 1947f03e107447bb8e23fe5deaf1ea60 (table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a]). Expected version 2 got 0
I20260808 13:35:25.586197 4836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 362dbd6990e740b39690537af98aa2f8
I20260808 13:35:25.587842 4836 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 269) took 1774 ms. Trace:
I20260808 13:35:25.592105 5609 tablet_service.cc:1514] Processing DeleteTablet for tablet 362dbd6990e740b39690537af98aa2f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:25.588186 4836 rpcz_store.cc:276] 0808 13:35:23.813052 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.187530 (+374478us) service_pool.cc:227] Handling call
0808 13:35:24.188978 (+ 1448us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:24.189517 (+ 539us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:24.189686 (+ 169us) catalog_manager.cc:2761] Locking tablets
0808 13:35:25.082577 (+892891us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:25.112403 (+ 29826us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:25.112700 (+ 297us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:25.586105 (+473405us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:25.587769 (+ 1664us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:25.085020 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:25.085987 (+ 967us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:25.086038 (+ 51us) write_op.cc:435] Acquired schema lock
0808 13:35:25.086075 (+ 37us) tablet.cc:660] Decoding operations
0808 13:35:25.086853 (+ 778us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:25.086971 (+ 118us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:25.087015 (+ 44us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:25.087671 (+ 656us) tablet.cc:699] Row locks acquired
0808 13:35:25.087720 (+ 49us) write_op.cc:260] PREPARE: finished
0808 13:35:25.087963 (+ 243us) write_op.cc:270] Start()
0808 13:35:25.088180 (+ 217us) write_op.cc:276] Timestamp: P: 1786196125087827 usec, L: 0
0808 13:35:25.088234 (+ 54us) op_driver.cc:331] REPLICATION: starting
0808 13:35:25.089442 (+ 1208us) log.cc:842] Serialized 725 byte log entry
0808 13:35:25.106518 (+ 17076us) op_driver.cc:444] REPLICATION: finished
0808 13:35:25.108353 (+ 1835us) write_op.cc:301] APPLY: starting
0808 13:35:25.108461 (+ 108us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:25.108744 (+ 283us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:25.108969 (+ 225us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:25.109607 (+ 638us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:25.110017 (+ 410us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:25.110163 (+ 146us) write_op.cc:312] APPLY: finished
0808 13:35:25.111679 (+ 1516us) log.cc:842] Serialized 29 byte log entry
0808 13:35:25.111934 (+ 255us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:25.112060 (+ 126us) write_op.cc:454] Released schema lock
0808 13:35:25.112121 (+ 61us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:25.112962 (+ 841us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:25.113029 (+ 67us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1581,"apply.run_cpu_time_us":3328,"apply.run_wall_time_us":33949,"num_ops":2,"prepare.queue_time_us":304,"prepare.run_cpu_time_us":5044,"prepare.run_wall_time_us":14713,"raft.queue_time_us":13529,"raft.run_cpu_time_us":1232,"raft.run_wall_time_us":1232,"replication_time_us":18193}]]}
I20260808 13:35:25.603091 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 362dbd6990e740b39690537af98aa2f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:25.569576 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 84580.8us
I20260808 13:35:25.603869 5384 raft_consensus.cc:2454] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:25.613045 7039 raft_consensus.cc:2726] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:25.615653 6796 leader_election.cc:288] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:25.543007 6913 leader_election.cc:288] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:25.632474 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e18e51c2e24d3caa3d29e6645e46ae" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:25.633366 5644 raft_consensus.cc:3037] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:25.634929 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c59eea10ec45af8e3a0992f9cc74bf" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:25.619273 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 108392us
I20260808 13:35:25.625797 5833 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:25 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:25.657554 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e18e51c2e24d3caa3d29e6645e46ae" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:25.658811 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 362dbd6990e740b39690537af98aa2f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:25.667158 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c59eea10ec45af8e3a0992f9cc74bf" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:25.668373 6033 tablet_bootstrap.cc:654] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:25.674939 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 201) took 2124 ms. Trace:
I20260808 13:35:25.675508 7013 rpcz_store.cc:276] 0808 13:35:23.550049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.550149 (+ 100us) service_pool.cc:227] Handling call
0808 13:35:25.023819 (+1473670us) ts_tablet_manager.cc:1145] Deleting tablet a0b29786cb404bc2b01473c3b8390a50
0808 13:35:25.023892 (+ 73us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.144902 (+121010us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:25.414341 (+269439us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:25.674830 (+260489us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":74188,"tablet-delete.queue_time_us":1472869}
I20260808 13:35:25.677780 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 202) took 2126 ms. Trace:
I20260808 13:35:25.684451 6046 tablet_bootstrap.cc:492] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:25.684190 5644 raft_consensus.cc:2454] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:25.684878 6046 ts_tablet_manager.cc:1397] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.550s user 0.026s sys 0.050s
W20260808 13:35:25.688028 5283 leader_election.cc:341] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:25.689603 6046 raft_consensus.cc:348] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.690251 6046 raft_consensus.cc:374] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:25.690339 4835 master_service.cc:946] ReplaceTablet: received request to replace tablet 543debff6da54d26bf62260bf8721ff7 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:25.690568 5285 leader_election.cc:302] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:25.691781 7039 raft_consensus.cc:2781] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:25.692186 7039 raft_consensus.cc:686] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:25.692891 7039 consensus_queue.cc:237] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:25.695669 4821 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet d09eb729339e47eb8db88eb6509daab4 (table default.Table_233ac5cc38504a5eb9b9cbf426cc19c7 [id=af41016671a644dcaacf0b2602d83672], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d09eb729339e47eb8db88eb6509daab4 consensus state has no leader
I20260808 13:35:25.690502 6046 raft_consensus.cc:729] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:25.699936 6026 tablet_bootstrap.cc:654] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:25.700392 6046 consensus_queue.cc:260] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.678210 7013 rpcz_store.cc:276] 0808 13:35:23.551352 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.551468 (+ 116us) service_pool.cc:227] Handling call
0808 13:35:25.677491 (+2126023us) ts_tablet_manager.cc:1145] Deleting tablet 60f8863bc32e49399940912444bcb972
0808 13:35:25.677557 (+ 66us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.677731 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2944,"tablet-delete.queue_time_us":2125437}
I20260808 13:35:25.706048 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_22ad421714294dac9da7cb4984aee8f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:25.706642 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 203) took 2107 ms. Trace:
I20260808 13:35:25.707039 7013 rpcz_store.cc:276] 0808 13:35:23.598991 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.599110 (+ 119us) service_pool.cc:227] Handling call
0808 13:35:25.706332 (+2107222us) ts_tablet_manager.cc:1145] Deleting tablet 6d8018bf8ff843d8818fbba9473acf47
0808 13:35:25.706380 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.706556 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2106583}
W20260808 13:35:25.708375 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22ad421714294dac9da7cb4984aee8f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:25.708820 7013 tablet_replica.cc:330] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:25.710100 7041 raft_consensus.cc:482] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:25.711126 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 60f8863bc32e49399940912444bcb972 because the tablet was not found. No further retry: Not found: Tablet not found: 60f8863bc32e49399940912444bcb972
I20260808 13:35:25.711172 7041 raft_consensus.cc:504] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:25.712698 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 6d8018bf8ff843d8818fbba9473acf47 because the tablet was not found. No further retry: Not found: Tablet not found: 6d8018bf8ff843d8818fbba9473acf47
I20260808 13:35:25.713760 7013 raft_consensus.cc:2229] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:25.748390 5283 leader_election.cc:302] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:25.754320 7013 raft_consensus.cc:2258] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:25.760412 6046 ts_tablet_manager.cc:1428] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.074s user 0.006s sys 0.000s
I20260808 13:35:25.761785 6046 tablet_bootstrap.cc:492] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:25.764122 7013 ts_tablet_manager.cc:1905] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:25.766369 7041 leader_election.cc:288] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:25.778038 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 123983us
I20260808 13:35:25.778874 7049 raft_consensus.cc:2726] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:25.794559 4835 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c1f683d6e76f42f086b0442473caa973" } new_table_name: "default.Table_1665f65d290446e79ac5f6550ec733b7" modify_external_catalogs: true
I20260808 13:35:25.803104 5383 raft_consensus.cc:3037] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:25.806156 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:25.808221 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 1947f03e107447bb8e23fe5deaf1ea60 (table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a]). Expected version 2 got 0
I20260808 13:35:25.809571 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet a8e18e51c2e24d3caa3d29e6645e46ae (table default.Table_321b4eb3dd024925af1cb354d3fe1567 [id=3c108c9e89424d9a9e11554601cf427c]). Expected version 1 got 0
I20260808 13:35:25.818167 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
W20260808 13:35:25.819470 5283 leader_election.cc:341] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: 81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:25.824787 6585 ts_tablet_manager.cc:1918] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:25.825152 6585 log.cc:1196] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/973a7afdcbf944aab8f11b3fd3995865
I20260808 13:35:25.825867 6585 ts_tablet_manager.cc:1939] T 973a7afdcbf944aab8f11b3fd3995865 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
W20260808 13:35:25.829684 5285 leader_election.cc:341] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:25.830235 5285 leader_election.cc:302] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:25.832304 6913 raft_consensus.cc:2726] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:25.837913 7014 ts_tablet_manager.cc:1918] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:25.838598 7014 log.cc:1196] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/f0c76b819474468baac51365360becfc
I20260808 13:35:25.839591 7014 ts_tablet_manager.cc:1939] T f0c76b819474468baac51365360becfc P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:25.845453 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 197) took 2228 ms. Trace:
I20260808 13:35:25.857438 6033 tablet_bootstrap.cc:492] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:25.857900 6033 ts_tablet_manager.cc:1397] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.378s user 0.004s sys 0.016s
I20260808 13:35:25.862098 6033 raft_consensus.cc:348] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.863034 6033 raft_consensus.cc:374] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:25.863298 6033 raft_consensus.cc:729] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:25.863960 6033 consensus_queue.cc:260] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.866217 6033 ts_tablet_manager.cc:1428] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260808 13:35:25.867208 6033 tablet_bootstrap.cc:492] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:25.896947 7094 consensus_queue.cc:1035] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:25.900534 6033 tablet_bootstrap.cc:654] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:25.909824 6913 consensus_queue.cc:1035] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:25.925377 6033 tablet_bootstrap.cc:492] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:25.926186 6033 ts_tablet_manager.cc:1397] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.059s user 0.006s sys 0.014s
I20260808 13:35:25.931296 6033 raft_consensus.cc:348] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.932227 6033 raft_consensus.cc:374] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:25.932596 6033 raft_consensus.cc:729] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:25.860917 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 973a7afdcbf944aab8f11b3fd3995865 (table default.Table_5c71e9adaaf5477b996f88813566c561 [id=eb20921cee2f48dc86300ce8ce5f52a1]) successfully deleted
W20260808 13:35:25.937644 4819 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 76755.1us
W20260808 13:35:25.938551 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 896389941179456d9d0ed3cd14726f11 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 896389941179456d9d0ed3cd14726f11 already in progress: creating tablet
W20260808 13:35:25.939395 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 2cd6945ecccc4684a3fa5b2a1f07057f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2cd6945ecccc4684a3fa5b2a1f07057f already in progress: creating tablet
W20260808 13:35:25.939924 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 362dbd6990e740b39690537af98aa2f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 362dbd6990e740b39690537af98aa2f8 already in progress: creating tablet
I20260808 13:35:25.936539 6033 consensus_queue.cc:260] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.941635 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:25.945796 6033 ts_tablet_manager.cc:1428] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.019s user 0.005s sys 0.001s
I20260808 13:35:25.947193 6033 tablet_bootstrap.cc:492] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:25.982579 7014 rpcz_store.cc:276] 0808 13:35:23.617033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.627383 (+ 10350us) service_pool.cc:227] Handling call
0808 13:35:24.760318 (+1132935us) ts_tablet_manager.cc:1145] Deleting tablet f0c76b819474468baac51365360becfc
0808 13:35:24.760408 (+ 90us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.518500 (+758092us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:25.837595 (+319095us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:25.845360 (+ 7765us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1132244}
I20260808 13:35:25.985360 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet f0c76b819474468baac51365360becfc (table default.Table_c4e53d8c393449429ef2791497ee7208 [id=26fb789fff434bb0825daf68d5378013]) successfully deleted
I20260808 13:35:25.986883 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 198) took 2358 ms. Trace:
W20260808 13:35:25.976992 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 88429.5us
I20260808 13:35:25.994849 6033 tablet_bootstrap.cc:654] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:25.996007 5882 raft_consensus.cc:3037] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:26.006465 6901 raft_consensus.cc:482] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:26.007395 6901 raft_consensus.cc:504] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.011533 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e992a0294c404cfb9541b57782a05fc9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:26.012944 5550 leader_election.cc:341] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:25.923576 6026 tablet_bootstrap.cc:492] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:26.018918 6026 ts_tablet_manager.cc:1397] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.585s user 0.016s sys 0.001s
I20260808 13:35:26.021803 6026 raft_consensus.cc:348] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.022464 6026 raft_consensus.cc:374] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.022778 6026 raft_consensus.cc:729] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.023495 6026 consensus_queue.cc:260] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:25.987529 7014 rpcz_store.cc:276] 0808 13:35:23.628371 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.628462 (+ 91us) service_pool.cc:227] Handling call
0808 13:35:25.986386 (+2357924us) ts_tablet_manager.cc:1145] Deleting tablet 2173e97590be43f9a24f55689d796269
0808 13:35:25.986470 (+ 84us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:25.986789 (+ 319us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2357408}
W20260808 13:35:26.025717 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 2173e97590be43f9a24f55689d796269 because the tablet was not found. No further retry: Not found: Tablet not found: 2173e97590be43f9a24f55689d796269
I20260808 13:35:26.027110 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 199) took 2397 ms. Trace:
I20260808 13:35:26.027647 7014 rpcz_store.cc:276] 0808 13:35:23.629249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.634693 (+ 5444us) service_pool.cc:227] Handling call
0808 13:35:26.026847 (+2392154us) ts_tablet_manager.cc:1145] Deleting tablet a0b29786cb404bc2b01473c3b8390a50
0808 13:35:26.026891 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.027055 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2391610}
I20260808 13:35:26.029798 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 201) took 2394 ms. Trace:
I20260808 13:35:26.030215 6913 raft_consensus.cc:482] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:26.030321 7014 rpcz_store.cc:276] 0808 13:35:23.635546 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.636634 (+ 1088us) service_pool.cc:227] Handling call
0808 13:35:26.029439 (+2392805us) ts_tablet_manager.cc:1145] Deleting tablet 196d80d806b0432cacf2ca6aa0da8c6b
0808 13:35:26.029484 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.029743 (+ 259us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2392227}
I20260808 13:35:26.030692 6913 raft_consensus.cc:504] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:26.031814 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 196d80d806b0432cacf2ca6aa0da8c6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 196d80d806b0432cacf2ca6aa0da8c6b already in progress: creating tablet
I20260808 13:35:26.032626 6913 leader_election.cc:288] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:26.032975 6033 tablet_bootstrap.cc:492] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:26.033427 6033 ts_tablet_manager.cc:1397] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.087s user 0.006s sys 0.013s
I20260808 13:35:26.034143 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd8f12cda94c4eb9a848f002a4020b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:26.035089 5636 raft_consensus.cc:2379] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:26.036036 6033 raft_consensus.cc:348] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.036830 6033 raft_consensus.cc:374] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.037102 6033 raft_consensus.cc:729] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.037701 6033 consensus_queue.cc:260] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.038347 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd8f12cda94c4eb9a848f002a4020b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:26.040248 6033 ts_tablet_manager.cc:1428] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 13:35:26.041302 6033 tablet_bootstrap.cc:492] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:26.051447 7094 consensus_queue.cc:1035] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:26.055023 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e992a0294c404cfb9541b57782a05fc9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:26.057137 5551 leader_election.cc:341] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:26.057690 5551 leader_election.cc:302] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:26.067269 5283 leader_election.cc:302] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:26.068804 7094 raft_consensus.cc:3037] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:26.076320 6033 tablet_bootstrap.cc:654] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:26.081597 4823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260808 13:35:26.112877 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 45787.9us
W20260808 13:35:26.124152 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 67042.5us
I20260808 13:35:26.125954 6026 ts_tablet_manager.cc:1428] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.107s user 0.006s sys 0.000s
I20260808 13:35:26.123726 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 212) took 1877 ms. Trace:
I20260808 13:35:26.136119 7014 rpcz_store.cc:276] 0808 13:35:24.183775 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.319224 (+135449us) service_pool.cc:227] Handling call
0808 13:35:26.032588 (+1713364us) ts_tablet_manager.cc:1145] Deleting tablet 86c2b34c95244472beec43643e60ff5f
0808 13:35:26.032634 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.060879 (+ 28245us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1535264}
I20260808 13:35:26.137249 7014 tablet_replica.cc:330] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:26.137931 6026 tablet_bootstrap.cc:492] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:26.143188 7014 raft_consensus.cc:2229] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:26.143771 7014 raft_consensus.cc:2258] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:26.139196 7032 consensus_queue.cc:1035] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260808 13:35:26.138447 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 86c2b34c95244472beec43643e60ff5f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 86c2b34c95244472beec43643e60ff5f already in progress: creating tablet
I20260808 13:35:26.138442 7125 raft_consensus.cc:2726] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:26.135551 6901 leader_election.cc:288] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:26.105369 6046 tablet_bootstrap.cc:654] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.172113 7014 ts_tablet_manager.cc:1905] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:26.176615 7039 raft_consensus.cc:482] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:26.177187 6026 tablet_bootstrap.cc:654] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.177433 7039 raft_consensus.cc:504] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.179948 7039 leader_election.cc:288] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:26.195019 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a541ec35b1a44109866fabc953f14cc6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:26.196103 5642 raft_consensus.cc:2454] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:26.197470 5285 leader_election.cc:302] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:26.200114 7049 raft_consensus.cc:2781] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:26.200443 7049 raft_consensus.cc:482] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:26.200773 7049 raft_consensus.cc:3037] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:26.195060 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 70230.1us
I20260808 13:35:26.204567 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a541ec35b1a44109866fabc953f14cc6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:26.205152 5877 raft_consensus.cc:2454] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:26.212823 7115 consensus_queue.cc:1035] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:26.221722 6033 tablet_bootstrap.cc:492] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:26.222121 6033 ts_tablet_manager.cc:1397] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.181s user 0.020s sys 0.036s
I20260808 13:35:26.223913 6026 tablet_bootstrap.cc:492] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:26.224308 6026 ts_tablet_manager.cc:1397] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.087s user 0.009s sys 0.000s
I20260808 13:35:26.224551 6033 raft_consensus.cc:348] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.225200 6033 raft_consensus.cc:374] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.225453 6033 raft_consensus.cc:729] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.181906 7094 raft_consensus.cc:2726] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:26.226588 7039 consensus_queue.cc:1035] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:26.226702 7049 raft_consensus.cc:504] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:26.228597 5283 consensus_peers.cc:487] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973a7afdcbf944aab8f11b3fd3995865. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:26.229652 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a541ec35b1a44109866fabc953f14cc6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:26.230055 7115 consensus_queue.cc:1035] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:26.230299 5644 raft_consensus.cc:3037] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:26.239358 5644 raft_consensus.cc:2454] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:26.240644 5285 leader_election.cc:302] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:26.241168 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a541ec35b1a44109866fabc953f14cc6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:26.241755 5878 raft_consensus.cc:3037] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:26.241811 7039 raft_consensus.cc:2781] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:26.245373 7039 raft_consensus.cc:686] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:26.246117 7039 consensus_queue.cc:237] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.249270 7049 leader_election.cc:288] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:26.226469 6026 raft_consensus.cc:348] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.251600 6026 raft_consensus.cc:374] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.251917 6026 raft_consensus.cc:729] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.252614 6026 consensus_queue.cc:260] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.249378 6046 tablet_bootstrap.cc:492] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:26.254834 6046 ts_tablet_manager.cc:1397] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.493s user 0.013s sys 0.000s
I20260808 13:35:26.259227 6046 raft_consensus.cc:348] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.259943 6046 raft_consensus.cc:374] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.284642 6026 ts_tablet_manager.cc:1428] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.060s user 0.006s sys 0.001s
I20260808 13:35:26.279637 6033 consensus_queue.cc:260] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.293365 7032 raft_consensus.cc:482] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:26.293915 7032 raft_consensus.cc:504] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.296069 7032 leader_election.cc:288] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:26.297866 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896389941179456d9d0ed3cd14726f11" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:26.300374 5550 leader_election.cc:341] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:26.303071 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:26.303712 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 1947f03e107447bb8e23fe5deaf1ea60 (table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a]). Expected version 2 got 0
I20260808 13:35:26.304944 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896389941179456d9d0ed3cd14726f11" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:26.306701 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet a8e18e51c2e24d3caa3d29e6645e46ae (table default.Table_321b4eb3dd024925af1cb354d3fe1567 [id=3c108c9e89424d9a9e11554601cf427c]). Expected version 1 got 0
W20260808 13:35:26.306717 5551 leader_election.cc:341] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:26.307111 5551 leader_election.cc:302] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:26.307996 7032 raft_consensus.cc:2726] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:26.310015 6026 tablet_bootstrap.cc:492] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:26.310624 5834 tablet_service.cc:1467] Processing CreateTablet for tablet 7c7e8060cab44121acf6dadd2c2873b3 (DEFAULT_TABLE table=default.Table_de3d9a6e588d44f698c656a21558e2ae [id=f92accd458dd439abf6c75dd1cd79df5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.312108 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c7e8060cab44121acf6dadd2c2873b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.315575 6033 ts_tablet_manager.cc:1428] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.093s user 0.002s sys 0.005s
I20260808 13:35:26.260180 6046 raft_consensus.cc:729] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.318760 7014 ts_tablet_manager.cc:1918] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:26.319326 7014 log.cc:1196] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/973a7afdcbf944aab8f11b3fd3995865
I20260808 13:35:26.320261 7014 ts_tablet_manager.cc:1939] T 973a7afdcbf944aab8f11b3fd3995865 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:26.309731 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:26.323904 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 7c7e8060cab44121acf6dadd2c2873b3 (DEFAULT_TABLE table=default.Table_de3d9a6e588d44f698c656a21558e2ae [id=f92accd458dd439abf6c75dd1cd79df5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.325652 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c7e8060cab44121acf6dadd2c2873b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.318140 6046 consensus_queue.cc:260] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.328157 6046 ts_tablet_manager.cc:1428] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.073s user 0.005s sys 0.000s
I20260808 13:35:26.328976 6046 tablet_bootstrap.cc:492] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:26.330987 7013 ts_tablet_manager.cc:1918] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:26.331374 7013 log.cc:1196] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/543debff6da54d26bf62260bf8721ff7
I20260808 13:35:26.332093 7013 ts_tablet_manager.cc:1939] T 543debff6da54d26bf62260bf8721ff7 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:26.334308 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 204) took 2734 ms. Trace:
I20260808 13:35:26.254005 5878 raft_consensus.cc:2454] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:26.340216 6033 tablet_bootstrap.cc:492] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:26.344481 6810 raft_consensus.cc:482] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:26.345077 6810 raft_consensus.cc:504] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.345506 5600 tablet_service.cc:1467] Processing CreateTablet for tablet 7c7e8060cab44121acf6dadd2c2873b3 (DEFAULT_TABLE table=default.Table_de3d9a6e588d44f698c656a21558e2ae [id=f92accd458dd439abf6c75dd1cd79df5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.347299 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c7e8060cab44121acf6dadd2c2873b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.349668 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1947f03e107447bb8e23fe5deaf1ea60" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:26.350736 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 213) took 1774 ms. Trace:
I20260808 13:35:26.351442 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1947f03e107447bb8e23fe5deaf1ea60" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:26.352398 6810 leader_election.cc:288] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:26.353458 5772 leader_election.cc:302] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:26.354818 7115 raft_consensus.cc:2726] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:26.351073 7014 rpcz_store.cc:276] 0808 13:35:24.576161 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.588329 (+ 12168us) service_pool.cc:227] Handling call
0808 13:35:26.137084 (+1548755us) ts_tablet_manager.cc:1145] Deleting tablet 973a7afdcbf944aab8f11b3fd3995865
0808 13:35:26.137129 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.192993 (+ 55864us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:26.271633 (+ 78640us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:26.350658 (+ 79025us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1547507}
I20260808 13:35:26.356421 7013 rpcz_store.cc:276] 0808 13:35:23.600241 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.600354 (+ 113us) service_pool.cc:227] Handling call
0808 13:35:25.708522 (+2108168us) ts_tablet_manager.cc:1145] Deleting tablet 543debff6da54d26bf62260bf8721ff7
0808 13:35:25.708615 (+ 93us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.235793 (+527178us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:26.330767 (+ 94974us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:26.334241 (+ 3474us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2107616}
I20260808 13:35:26.357818 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 543debff6da54d26bf62260bf8721ff7 (table default.Table_062151f3d1444efc8a42ba6cd06e19c9 [id=68dee345bd2f4fc1823048b3da8d944f]) successfully deleted
I20260808 13:35:26.358640 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 973a7afdcbf944aab8f11b3fd3995865 (table default.Table_5c71e9adaaf5477b996f88813566c561 [id=eb20921cee2f48dc86300ce8ce5f52a1]) successfully deleted
I20260808 13:35:26.358819 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 205) took 2757 ms. Trace:
I20260808 13:35:26.359105 7013 rpcz_store.cc:276] 0808 13:35:23.601239 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.601342 (+ 103us) service_pool.cc:227] Handling call
0808 13:35:26.358524 (+2757182us) ts_tablet_manager.cc:1145] Deleting tablet e7616d81aa924933a9b24d44004725fa
0808 13:35:26.358571 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.358765 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2756696}
I20260808 13:35:26.359961 7014 tablet_replica.cc:330] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:26.360497 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
W20260808 13:35:26.360680 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet e7616d81aa924933a9b24d44004725fa because the tablet was not found. No further retry: Not found: Tablet not found: e7616d81aa924933a9b24d44004725fa
I20260808 13:35:26.361928 6026 tablet_bootstrap.cc:654] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.370002 7049 raft_consensus.cc:482] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:26.370431 7049 raft_consensus.cc:504] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.372284 7049 leader_election.cc:288] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:26.372992 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 3272708a835c432c878cc330e44490c5 (DEFAULT_TABLE table=default.Table_ca3570782c0f48a88952f5f442fdbde6 [id=654d6eaf69f34f4ca5a5e7e1d1480df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.374125 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3272708a835c432c878cc330e44490c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.374162 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ea98afef6841fead641b314ee53a6e" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:26.374754 5642 raft_consensus.cc:2454] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:26.375799 6026 tablet_bootstrap.cc:492] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:26.376163 6026 ts_tablet_manager.cc:1397] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.067s user 0.019s sys 0.001s
I20260808 13:35:26.375979 5285 leader_election.cc:302] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:26.377151 7049 raft_consensus.cc:2781] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:26.377513 7049 raft_consensus.cc:482] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:26.378082 7049 raft_consensus.cc:3037] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:26.379881 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ea98afef6841fead641b314ee53a6e" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:26.380277 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 3272708a835c432c878cc330e44490c5 (DEFAULT_TABLE table=default.Table_ca3570782c0f48a88952f5f442fdbde6 [id=654d6eaf69f34f4ca5a5e7e1d1480df6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:26.382201 5283 leader_election.cc:341] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:26.386430 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 3272708a835c432c878cc330e44490c5 (DEFAULT_TABLE table=default.Table_ca3570782c0f48a88952f5f442fdbde6 [id=654d6eaf69f34f4ca5a5e7e1d1480df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.387661 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3272708a835c432c878cc330e44490c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.388298 6033 tablet_bootstrap.cc:654] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.388453 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 20113491b108476b95b8f32b69176d0e (DEFAULT_TABLE table=default.Table_feb9646b81ef46c7aa00ea076ae7240f [id=d4e8425d1b414ba3b4480306662579a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.390271 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3272708a835c432c878cc330e44490c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.391402 6026 raft_consensus.cc:348] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.392061 6026 raft_consensus.cc:374] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.392290 6026 raft_consensus.cc:729] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.393913 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20113491b108476b95b8f32b69176d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.362054 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 206) took 2759 ms. Trace:
I20260808 13:35:26.403357 7013 rpcz_store.cc:276] 0808 13:35:23.602157 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.602245 (+ 88us) service_pool.cc:227] Handling call
0808 13:35:26.361605 (+2759360us) ts_tablet_manager.cc:1145] Deleting tablet cfc9a6ec9a86431fb46e50fcaa658d33
0808 13:35:26.361677 (+ 72us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.361971 (+ 294us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2758817}
I20260808 13:35:26.405220 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 207) took 2802 ms. Trace:
W20260808 13:35:26.405565 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet cfc9a6ec9a86431fb46e50fcaa658d33 because the tablet was not found. No further retry: Not found: Tablet not found: cfc9a6ec9a86431fb46e50fcaa658d33
I20260808 13:35:26.407512 6046 tablet_bootstrap.cc:654] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.407956 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 20113491b108476b95b8f32b69176d0e (DEFAULT_TABLE table=default.Table_feb9646b81ef46c7aa00ea076ae7240f [id=d4e8425d1b414ba3b4480306662579a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.409140 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20113491b108476b95b8f32b69176d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.413621 7014 raft_consensus.cc:2229] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:26.422282 6026 consensus_queue.cc:260] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.422979 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 20113491b108476b95b8f32b69176d0e (DEFAULT_TABLE table=default.Table_feb9646b81ef46c7aa00ea076ae7240f [id=d4e8425d1b414ba3b4480306662579a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.424939 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20113491b108476b95b8f32b69176d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.429577 6026 ts_tablet_manager.cc:1428] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.053s user 0.010s sys 0.003s
I20260808 13:35:26.405761 7013 rpcz_store.cc:276] 0808 13:35:23.603102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.603192 (+ 90us) service_pool.cc:227] Handling call
0808 13:35:26.404942 (+2801750us) ts_tablet_manager.cc:1145] Deleting tablet 345d13cbde6844079c2d1eb80732bb1f
0808 13:35:26.404986 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.405166 (+ 180us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2801186}
I20260808 13:35:26.372256 4841 catalog_manager.cc:5495] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:26.433943 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 208) took 2829 ms. Trace:
I20260808 13:35:26.434031 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:26.434276 7013 rpcz_store.cc:276] 0808 13:35:23.604161 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.604266 (+ 105us) service_pool.cc:227] Handling call
0808 13:35:26.433633 (+2829367us) ts_tablet_manager.cc:1145] Deleting tablet 78d492406c8a4c4186131e02a1559cb4
0808 13:35:26.433678 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.433884 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2828702}
I20260808 13:35:26.435920 4835 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 280) took 1176 ms. Trace:
I20260808 13:35:26.436268 6033 tablet_bootstrap.cc:492] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:26.436697 6033 ts_tablet_manager.cc:1397] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.109s user 0.010s sys 0.005s
W20260808 13:35:26.435935 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 345d13cbde6844079c2d1eb80732bb1f because the tablet was not found. No further retry: Not found: Tablet not found: 345d13cbde6844079c2d1eb80732bb1f
I20260808 13:35:26.438840 6033 raft_consensus.cc:348] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.439457 6033 raft_consensus.cc:374] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260808 13:35:26.439648 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 78d492406c8a4c4186131e02a1559cb4 because the tablet was not found. No further retry: Not found: Tablet not found: 78d492406c8a4c4186131e02a1559cb4
I20260808 13:35:26.439685 6033 raft_consensus.cc:729] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.440336 6033 consensus_queue.cc:260] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.441874 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 209) took 2790 ms. Trace:
I20260808 13:35:26.442233 6033 ts_tablet_manager.cc:1428] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 13:35:26.436385 4835 rpcz_store.cc:276] 0808 13:35:25.259186 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:25.776120 (+516934us) service_pool.cc:227] Handling call
0808 13:35:25.794460 (+ 18340us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff7d6 00007f227bd45634 00007f227bd455c2 00007f227bd454f1 00007f227bd4546c 00007f227bd44722 00007f2279fde94c 00007f2279fddd86
0808 13:35:25.796376 (+ 1916us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:25.797194 (+ 818us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:25.797894 (+ 700us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:25.798126 (+ 232us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:25.927962 (+129836us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:26.435834 (+507872us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:25.912352 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:25.913077 (+ 725us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:25.913105 (+ 28us) write_op.cc:435] Acquired schema lock
0808 13:35:25.913120 (+ 15us) tablet.cc:660] Decoding operations
0808 13:35:25.913430 (+ 310us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:25.913479 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:25.913503 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:25.914014 (+ 511us) tablet.cc:699] Row locks acquired
0808 13:35:25.914027 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:25.914151 (+ 124us) write_op.cc:270] Start()
0808 13:35:25.914249 (+ 98us) write_op.cc:276] Timestamp: P: 1786196125914081 usec, L: 0
0808 13:35:25.914267 (+ 18us) op_driver.cc:331] REPLICATION: starting
0808 13:35:25.914980 (+ 713us) log.cc:842] Serialized 405 byte log entry
0808 13:35:25.925034 (+ 10054us) op_driver.cc:444] REPLICATION: finished
0808 13:35:25.925405 (+ 371us) write_op.cc:301] APPLY: starting
0808 13:35:25.925457 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:25.925549 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:25.925646 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:25.925876 (+ 230us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:25.926004 (+ 128us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:25.926043 (+ 39us) write_op.cc:312] APPLY: finished
0808 13:35:25.926218 (+ 175us) log.cc:842] Serialized 23 byte log entry
0808 13:35:25.926372 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:25.926482 (+ 110us) write_op.cc:454] Released schema lock
0808 13:35:25.926512 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:25.926915 (+ 403us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:25.926945 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":40484608,"child_traces":[["op",{"apply.queue_time_us":210,"apply.run_cpu_time_us":1512,"apply.run_wall_time_us":1703,"num_ops":1,"prepare.queue_time_us":111036,"prepare.run_cpu_time_us":2759,"prepare.run_wall_time_us":62408,"raft.queue_time_us":110,"raft.run_cpu_time_us":804,"raft.run_wall_time_us":1734,"replication_time_us":10689}]]}
I20260808 13:35:26.449939 7115 consensus_queue.cc:1035] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:26.450803 6026 tablet_bootstrap.cc:492] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:26.442308 7013 rpcz_store.cc:276] 0808 13:35:23.651058 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.651219 (+ 161us) service_pool.cc:227] Handling call
0808 13:35:26.441255 (+2790036us) ts_tablet_manager.cc:1145] Deleting tablet 5b6e6ce417cc45fd90f5858c30d37a47
0808 13:35:26.441319 (+ 64us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.441791 (+ 472us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2789354}
W20260808 13:35:26.455562 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 5b6e6ce417cc45fd90f5858c30d37a47 because the tablet was not found. No further retry: Not found: Tablet not found: 5b6e6ce417cc45fd90f5858c30d37a47
I20260808 13:35:26.459576 7013 tablet_replica.cc:330] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:26.461030 7013 raft_consensus.cc:2229] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:26.475502 7013 raft_consensus.cc:2258] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:26.482590 6033 tablet_bootstrap.cc:492] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:26.475325 6026 tablet_bootstrap.cc:654] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.487478 4833 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_13bc8e545bba490b8fef6864a18684a0 [id=d9baa62ea5c249a898586a45f8b4f372] alter complete (version 1)
I20260808 13:35:26.414181 7014 raft_consensus.cc:2258] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:26.493111 7014 ts_tablet_manager.cc:1905] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:26.496321 7016 consensus_queue.cc:1035] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:26.502744 7049 raft_consensus.cc:504] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.507647 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ea98afef6841fead641b314ee53a6e" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:26.508474 5877 raft_consensus.cc:3037] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:26.522478 6033 tablet_bootstrap.cc:654] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.526471 5877 raft_consensus.cc:2454] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:26.532871 4838 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 266) took 3992 ms. Trace:
I20260808 13:35:26.533445 7049 leader_election.cc:288] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:26.535784 5283 leader_election.cc:302] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:26.533346 4838 rpcz_store.cc:276] 0808 13:35:22.540662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.070146 (+1529484us) service_pool.cc:227] Handling call
0808 13:35:24.159642 (+ 89496us) spinlock_profiling.cc:243] Waited 87.6 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff7d6 00007f227bd45634 00007f227bd455c2 00007f227bd454f1 00007f227bd4546c 00007f227bd44722 00007f2279fde94c 00007f2279fddd86
0808 13:35:24.190351 (+ 30709us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:24.191887 (+ 1536us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:24.192599 (+ 712us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:24.192848 (+ 249us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:24.797852 (+605004us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:26.532789 (+1734937us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:24.489134 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:24.489876 (+ 742us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:24.489931 (+ 55us) write_op.cc:435] Acquired schema lock
0808 13:35:24.489976 (+ 45us) tablet.cc:660] Decoding operations
0808 13:35:24.490720 (+ 744us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:24.490806 (+ 86us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:24.490860 (+ 54us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:24.491233 (+ 373us) tablet.cc:699] Row locks acquired
0808 13:35:24.491285 (+ 52us) write_op.cc:260] PREPARE: finished
0808 13:35:24.491514 (+ 229us) write_op.cc:270] Start()
0808 13:35:24.491696 (+ 182us) write_op.cc:276] Timestamp: P: 1786196124491408 usec, L: 0
0808 13:35:24.491767 (+ 71us) op_driver.cc:331] REPLICATION: starting
0808 13:35:24.492799 (+ 1032us) log.cc:842] Serialized 405 byte log entry
0808 13:35:24.784690 (+291891us) op_driver.cc:444] REPLICATION: finished
0808 13:35:24.796512 (+ 11822us) write_op.cc:301] APPLY: starting
0808 13:35:24.796681 (+ 169us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:24.796789 (+ 108us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:24.796901 (+ 112us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:24.797099 (+ 198us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:24.797227 (+ 128us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:24.797286 (+ 59us) write_op.cc:312] APPLY: finished
0808 13:35:24.797461 (+ 175us) log.cc:842] Serialized 23 byte log entry
0808 13:35:24.797588 (+ 127us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.797637 (+ 49us) write_op.cc:454] Released schema lock
0808 13:35:24.797667 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:24.798094 (+ 427us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.798123 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":201427840,"child_traces":[["op",{"apply.queue_time_us":11637,"apply.run_cpu_time_us":1438,"apply.run_wall_time_us":1781,"num_ops":1,"prepare.queue_time_us":295056,"prepare.run_cpu_time_us":4189,"prepare.run_wall_time_us":56942,"raft.queue_time_us":21628,"raft.run_cpu_time_us":15,"raft.run_wall_time_us":16,"replication_time_us":292850}]]}
I20260808 13:35:26.537035 7049 raft_consensus.cc:2781] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:26.537488 7049 raft_consensus.cc:686] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:26.538446 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6ea0a150e9a245babc374b7222ad1ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:26.538348 7049 consensus_queue.cc:237] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:26.540114 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ea0a150e9a245babc374b7222ad1ca7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:26.550024 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ea98afef6841fead641b314ee53a6e" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:26.550599 5642 raft_consensus.cc:3037] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:26.552872 6033 tablet_bootstrap.cc:492] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:26.553273 6033 ts_tablet_manager.cc:1397] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.071s user 0.008s sys 0.003s
I20260808 13:35:26.556174 7016 raft_consensus.cc:482] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:26.556792 7016 raft_consensus.cc:504] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.561488 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e18e51c2e24d3caa3d29e6645e46ae" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:26.562676 5636 raft_consensus.cc:2379] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:26.564553 7016 leader_election.cc:288] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:26.564795 6033 raft_consensus.cc:348] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.565541 6033 raft_consensus.cc:374] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.565785 6033 raft_consensus.cc:729] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.566113 7013 ts_tablet_manager.cc:1905] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:26.566255 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e18e51c2e24d3caa3d29e6645e46ae" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:26.569502 5771 leader_election.cc:302] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:26.573835 7016 raft_consensus.cc:3037] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:26.583801 6033 consensus_queue.cc:260] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.584587 7115 raft_consensus.cc:482] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:26.585027 7115 raft_consensus.cc:504] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.590855 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896389941179456d9d0ed3cd14726f11" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:26.592571 5772 leader_election.cc:341] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: Tablet not RUNNING: STOPPED
I20260808 13:35:26.594854 7115 leader_election.cc:288] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:26.595592 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896389941179456d9d0ed3cd14726f11" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:26.597307 5771 leader_election.cc:341] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:26.597718 5771 leader_election.cc:302] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:26.598510 6810 raft_consensus.cc:2726] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:26.606343 5882 raft_consensus.cc:3037] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:26.612700 6033 ts_tablet_manager.cc:1428] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.059s user 0.005s sys 0.003s
I20260808 13:35:26.613816 6033 tablet_bootstrap.cc:492] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:26.617403 7014 ts_tablet_manager.cc:1918] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:26.617539 6046 tablet_bootstrap.cc:492] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:26.617833 7014 log.cc:1196] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/896389941179456d9d0ed3cd14726f11
I20260808 13:35:26.618252 6046 ts_tablet_manager.cc:1397] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.289s user 0.012s sys 0.003s
I20260808 13:35:26.593502 5642 raft_consensus.cc:2454] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:26.618497 7014 ts_tablet_manager.cc:1939] T 896389941179456d9d0ed3cd14726f11 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:26.629067 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 214) took 1866 ms. Trace:
W20260808 13:35:26.636533 5285 consensus_peers.cc:487] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973a7afdcbf944aab8f11b3fd3995865. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:26.629456 7014 rpcz_store.cc:276] 0808 13:35:24.762441 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.769592 (+ 7151us) service_pool.cc:227] Handling call
0808 13:35:26.359700 (+1590108us) ts_tablet_manager.cc:1145] Deleting tablet 896389941179456d9d0ed3cd14726f11
0808 13:35:26.359789 (+ 89us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.537674 (+177885us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:26.617151 (+ 79477us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:26.629002 (+ 11851us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1589409}
I20260808 13:35:26.644176 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 896389941179456d9d0ed3cd14726f11 (table default.Table_41272f24d0a5431b9f718128ad86cb81 [id=d7c7a7f08fa543aaa0bca2e996d1ee8e]) successfully deleted
I20260808 13:35:26.651827 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 218) took 1119 ms. Trace:
I20260808 13:35:26.660682 6033 tablet_bootstrap.cc:654] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.654940 7014 rpcz_store.cc:276] 0808 13:35:25.532047 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:25.532619 (+ 572us) service_pool.cc:227] Handling call
0808 13:35:26.645429 (+1112810us) ts_tablet_manager.cc:1145] Deleting tablet 2cd6945ecccc4684a3fa5b2a1f07057f
0808 13:35:26.645825 (+ 396us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.651752 (+ 5927us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1103254}
W20260808 13:35:26.661918 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 2cd6945ecccc4684a3fa5b2a1f07057f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2cd6945ecccc4684a3fa5b2a1f07057f already in progress: creating tablet
I20260808 13:35:26.663309 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 219) took 1071 ms. Trace:
I20260808 13:35:26.672550 7014 rpcz_store.cc:276] 0808 13:35:25.591502 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:25.591680 (+ 178us) service_pool.cc:227] Handling call
0808 13:35:26.662891 (+1071211us) ts_tablet_manager.cc:1145] Deleting tablet 362dbd6990e740b39690537af98aa2f8
0808 13:35:26.662945 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.663232 (+ 287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1070236}
W20260808 13:35:26.674506 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 362dbd6990e740b39690537af98aa2f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 362dbd6990e740b39690537af98aa2f8 already in progress: creating tablet
I20260808 13:35:26.683151 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 4625b632d67244fc86e48d739dde707e (table default.Table_1665f65d290446e79ac5f6550ec733b7 [id=2f47077a0fd6488ebc33ab8794c0a5b6]). Expected version 1 got 0
I20260808 13:35:26.685863 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 007c7222c7bc48329420c740b31ddfae (DEFAULT_TABLE table=default.Table_d5bb15ded3d442699c050da99887a456 [id=67953edd9d3c47b8b489f036a64c74ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.690172 6033 tablet_bootstrap.cc:492] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:26.690583 6033 ts_tablet_manager.cc:1397] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.077s user 0.009s sys 0.002s
I20260808 13:35:26.696342 6026 tablet_bootstrap.cc:492] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:26.696755 6026 ts_tablet_manager.cc:1397] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.246s user 0.028s sys 0.007s
I20260808 13:35:26.698977 6026 raft_consensus.cc:348] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.699685 6026 raft_consensus.cc:374] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.699915 6026 raft_consensus.cc:729] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.700769 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 007c7222c7bc48329420c740b31ddfae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.689347 7175 tablet.cc:1722] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:26.703132 6026 consensus_queue.cc:260] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.707854 6026 ts_tablet_manager.cc:1428] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20260808 13:35:26.702257 6033 raft_consensus.cc:348] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.711127 6033 raft_consensus.cc:374] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.711459 6033 raft_consensus.cc:729] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.712148 6033 consensus_queue.cc:260] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.686017 4835 catalog_manager.cc:5495] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:26.718699 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 007c7222c7bc48329420c740b31ddfae (DEFAULT_TABLE table=default.Table_d5bb15ded3d442699c050da99887a456 [id=67953edd9d3c47b8b489f036a64c74ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.719722 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 007c7222c7bc48329420c740b31ddfae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.720794 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 8d12106f0d794a79935999a2b8dbe7e9 (DEFAULT_TABLE table=default.Table_dae35fadbf624671ad716102b5182446 [id=054085276211478a8982d48b0599a684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.721607 5609 tablet_service.cc:1467] Processing CreateTablet for tablet 8d12106f0d794a79935999a2b8dbe7e9 (DEFAULT_TABLE table=default.Table_dae35fadbf624671ad716102b5182446 [id=054085276211478a8982d48b0599a684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.722141 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d12106f0d794a79935999a2b8dbe7e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.722527 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d12106f0d794a79935999a2b8dbe7e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.735185 7173 tablet.cc:1722] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:26.742055 6026 tablet_bootstrap.cc:492] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:26.750659 6033 ts_tablet_manager.cc:1428] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.060s user 0.004s sys 0.001s
I20260808 13:35:26.713732 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:26.751641 6033 tablet_bootstrap.cc:492] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:26.752594 4835 catalog_manager.cc:5495] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:26.753916 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet a8e18e51c2e24d3caa3d29e6645e46ae (table default.Table_321b4eb3dd024925af1cb354d3fe1567 [id=3c108c9e89424d9a9e11554601cf427c]). Expected version 1 got 0
I20260808 13:35:26.653896 7016 raft_consensus.cc:2726] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:26.776082 6033 tablet_bootstrap.cc:654] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.778659 6026 tablet_bootstrap.cc:654] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.779073 5832 tablet_service.cc:1467] Processing CreateTablet for tablet 42218e4330d147f389717108bfe0acea (DEFAULT_TABLE table=default.Table_9769e23d59f2498e925cd703900ecbb3 [id=f603e442844e4a09bfecb79b93014265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.780253 5832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42218e4330d147f389717108bfe0acea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.781235 5606 tablet_service.cc:1467] Processing CreateTablet for tablet 42218e4330d147f389717108bfe0acea (DEFAULT_TABLE table=default.Table_9769e23d59f2498e925cd703900ecbb3 [id=f603e442844e4a09bfecb79b93014265]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:26.794165 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 43623.1us
I20260808 13:35:26.798581 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_7a1963a007954f8ca1d330f9d1936dd3 [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 3 got 2
I20260808 13:35:26.799433 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:26.800297 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 1947f03e107447bb8e23fe5deaf1ea60 (table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a]). Expected version 2 got 0
I20260808 13:35:26.801191 4835 catalog_manager.cc:5495] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:26.806522 6026 tablet_bootstrap.cc:492] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:26.807021 7094 consensus_queue.cc:1035] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:26.809966 6026 ts_tablet_manager.cc:1397] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.068s user 0.014s sys 0.000s
I20260808 13:35:26.814285 6033 tablet_bootstrap.cc:492] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:26.814738 6033 ts_tablet_manager.cc:1397] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.004s
I20260808 13:35:26.813467 6026 raft_consensus.cc:348] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.823127 6026 raft_consensus.cc:374] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.823400 6026 raft_consensus.cc:729] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.824019 6026 consensus_queue.cc:260] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.827888 6046 raft_consensus.cc:348] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.828773 6046 raft_consensus.cc:374] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.829078 6046 raft_consensus.cc:729] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.829332 7115 raft_consensus.cc:482] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:26.829824 7115 raft_consensus.cc:504] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.831034 6033 raft_consensus.cc:348] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.831709 6033 raft_consensus.cc:374] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:26.831964 6033 raft_consensus.cc:729] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:26.829236 6026 ts_tablet_manager.cc:1428] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260808 13:35:26.833815 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42218e4330d147f389717108bfe0acea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.834929 6953 raft_consensus.cc:482] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:26.834328 6033 consensus_queue.cc:260] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.835335 6953 raft_consensus.cc:504] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.836961 6033 ts_tablet_manager.cc:1428] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260808 13:35:26.837965 6033 tablet_bootstrap.cc:492] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:26.839483 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e992a0294c404cfb9541b57782a05fc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:26.840348 5636 raft_consensus.cc:2454] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:26.841806 5772 leader_election.cc:302] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:26.844007 6026 tablet_bootstrap.cc:492] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:26.843706 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3682c9c656c4556bc13ae3aeddd7c5e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:26.844820 5644 raft_consensus.cc:2454] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:26.846343 5772 leader_election.cc:302] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:26.829823 6046 consensus_queue.cc:260] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.889401 7016 raft_consensus.cc:2781] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:26.889932 7016 raft_consensus.cc:482] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:26.890476 7016 raft_consensus.cc:3037] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:26.894912 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 48657.5us
I20260808 13:35:26.898448 6953 leader_election.cc:288] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:26.906893 6810 raft_consensus.cc:2781] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:26.907464 6810 raft_consensus.cc:482] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:26.907966 6810 raft_consensus.cc:3037] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:26.832217 7115 leader_election.cc:288] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:26.902087 5334 tablet_service.cc:1467] Processing CreateTablet for tablet 42218e4330d147f389717108bfe0acea (DEFAULT_TABLE table=default.Table_9769e23d59f2498e925cd703900ecbb3 [id=f603e442844e4a09bfecb79b93014265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.925211 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42218e4330d147f389717108bfe0acea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.907992 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3682c9c656c4556bc13ae3aeddd7c5e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:26.927205 7016 raft_consensus.cc:504] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.910460 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e992a0294c404cfb9541b57782a05fc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:26.869095 6046 ts_tablet_manager.cc:1428] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.251s user 0.003s sys 0.005s
I20260808 13:35:26.883700 7094 consensus_queue.cc:1035] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.052s
I20260808 13:35:26.939540 6026 tablet_bootstrap.cc:654] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.943392 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 007c7222c7bc48329420c740b31ddfae (DEFAULT_TABLE table=default.Table_d5bb15ded3d442699c050da99887a456 [id=67953edd9d3c47b8b489f036a64c74ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.944793 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 007c7222c7bc48329420c740b31ddfae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.930109 6033 tablet_bootstrap.cc:654] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:26.956023 6810 raft_consensus.cc:504] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:26.965641 5832 tablet_service.cc:1467] Processing CreateTablet for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f (DEFAULT_TABLE table=default.Table_aa07254565ac49d4aadd887511d6838b [id=14177aacb9124b618a01b8e2ff32abb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:26.967136 5832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:26.951264 6046 tablet_bootstrap.cc:492] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:26.915622 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 8d12106f0d794a79935999a2b8dbe7e9 (DEFAULT_TABLE table=default.Table_dae35fadbf624671ad716102b5182446 [id=054085276211478a8982d48b0599a684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.002723 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d12106f0d794a79935999a2b8dbe7e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.014773 7032 raft_consensus.cc:482] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:27.014771 7192 tablet.cc:1722] T 66d985de8de040829569d1f5783c6fef P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:27.015367 7032 raft_consensus.cc:504] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.016628 7175 tablet.cc:1722] T 66d985de8de040829569d1f5783c6fef P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:27.018914 5833 tablet_service.cc:1467] Processing CreateTablet for tablet b314734698ed423ba9cf4134db5846ec (DEFAULT_TABLE table=default.Table_2df92438979447cc89b7a5301a13d6a9 [id=40cafe491872437cba9d7294fff5519a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.019414 7032 leader_election.cc:288] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:27.020676 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b314734698ed423ba9cf4134db5846ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.021461 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234591e698f8472e90e1f610da37ff3a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:27.022879 5550 leader_election.cc:341] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:27.011693 7193 tablet.cc:1722] T 66d985de8de040829569d1f5783c6fef P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260808 13:35:27.028731 5771 leader_election.cc:341] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:27.029861 5771 leader_election.cc:341] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:27.037066 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234591e698f8472e90e1f610da37ff3a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:27.037930 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f (DEFAULT_TABLE table=default.Table_aa07254565ac49d4aadd887511d6838b [id=14177aacb9124b618a01b8e2ff32abb7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:27.038767 5383 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s user 0.012s sys 0.006s
I20260808 13:35:27.039168 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:27.039135 5383 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s user 0.012s sys 0.006s
I20260808 13:35:27.038792 5336 tablet_service.cc:1467] Processing CreateTablet for tablet b314734698ed423ba9cf4134db5846ec (DEFAULT_TABLE table=default.Table_2df92438979447cc89b7a5301a13d6a9 [id=40cafe491872437cba9d7294fff5519a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.044839 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b314734698ed423ba9cf4134db5846ec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:26.805785 4821 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet f0c76b819474468baac51365360becfc (table default.Table_c4e53d8c393449429ef2791497ee7208 [id=26fb789fff434bb0825daf68d5378013], current schema version=5) failed: Not found: failed to reset TS proxy: No leader found: Tablet f0c76b819474468baac51365360becfc consensus state has no leader
I20260808 13:35:26.877633 4838 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 281) took 1601 ms. Trace:
W20260808 13:35:27.048352 5551 leader_election.cc:341] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:27.048746 5551 leader_election.cc:302] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:27.049695 6901 raft_consensus.cc:2726] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:27.046988 7032 raft_consensus.cc:482] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:27.050453 7032 raft_consensus.cc:504] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.048484 4838 rpcz_store.cc:276] 0808 13:35:25.276554 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:26.538005 (+1261451us) service_pool.cc:227] Handling call
0808 13:35:26.540553 (+ 2548us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:26.553855 (+ 13302us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:26.876869 (+323014us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:26.877225 (+ 356us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:26.877550 (+ 325us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:26.563154 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:26.563982 (+ 828us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:26.564018 (+ 36us) write_op.cc:435] Acquired schema lock
0808 13:35:26.564722 (+ 704us) tablet.cc:660] Decoding operations
0808 13:35:26.571880 (+ 7158us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:26.571963 (+ 83us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:26.572016 (+ 53us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:26.572492 (+ 476us) tablet.cc:699] Row locks acquired
0808 13:35:26.572523 (+ 31us) write_op.cc:260] PREPARE: finished
0808 13:35:26.572790 (+ 267us) write_op.cc:270] Start()
0808 13:35:26.573019 (+ 229us) write_op.cc:276] Timestamp: P: 1786196126572700 usec, L: 0
0808 13:35:26.573072 (+ 53us) op_driver.cc:331] REPLICATION: starting
0808 13:35:26.574072 (+ 1000us) log.cc:842] Serialized 433 byte log entry
0808 13:35:26.872308 (+298236us) op_driver.cc:444] REPLICATION: finished
0808 13:35:26.872751 (+ 443us) write_op.cc:301] APPLY: starting
0808 13:35:26.872839 (+ 88us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:26.873080 (+ 241us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:26.873255 (+ 175us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:26.873988 (+ 733us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:26.874204 (+ 216us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:26.874293 (+ 89us) write_op.cc:312] APPLY: finished
0808 13:35:26.874658 (+ 365us) log.cc:842] Serialized 29 byte log entry
0808 13:35:26.874979 (+ 321us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:26.875121 (+ 142us) write_op.cc:454] Released schema lock
0808 13:35:26.875165 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:26.875548 (+ 383us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:26.875602 (+ 54us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":152,"apply.run_cpu_time_us":3183,"apply.run_wall_time_us":3191,"num_ops":2,"prepare.queue_time_us":7717,"prepare.run_cpu_time_us":4579,"prepare.run_wall_time_us":38315,"raft.queue_time_us":26561,"raft.run_cpu_time_us":16,"raft.run_wall_time_us":16,"replication_time_us":299099}]]}
I20260808 13:35:27.057464 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6826e4e4db2043b3a4fe9e7a9359cab6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:27.058234 5383 raft_consensus.cc:2454] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:27.059008 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e992a0294c404cfb9541b57782a05fc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:27.058506 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3682c9c656c4556bc13ae3aeddd7c5e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:27.051921 4833 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_13ca344305b74220b866f3194259e283 [id=16cef8437603476b97a8562edba58a41] alter complete (version 3)
I20260808 13:35:27.069058 5611 tablet_service.cc:1467] Processing CreateTablet for tablet b314734698ed423ba9cf4134db5846ec (DEFAULT_TABLE table=default.Table_2df92438979447cc89b7a5301a13d6a9 [id=40cafe491872437cba9d7294fff5519a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.070115 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b314734698ed423ba9cf4134db5846ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.076359 5610 tablet_service.cc:1514] Processing DeleteTablet for tablet 9260370cf8cb46a7b699581e4e8d46db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.077270 7014 tablet_replica.cc:330] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:27.077541 5610 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.078485 7014 raft_consensus.cc:2229] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:27.078748 5610 tablet_service.cc:1514] Processing DeleteTablet for tablet b809b185ee9e4c658a050c41c9abc4ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.079072 7014 raft_consensus.cc:2258] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:27.081359 7014 ts_tablet_manager.cc:1905] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:27.085485 4842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 38) took 2645 ms (client timeout 1000 ms). Trace:
W20260808 13:35:27.085839 4842 rpcz_store.cc:269] 0808 13:35:24.439867 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.582164 (+142297us) service_pool.cc:227] Handling call
0808 13:35:24.791136 (+208972us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:27.085404 (+2294268us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":24,"spinlock_wait_cycles":458107264}
W20260808 13:35:27.093044 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 37) took 3673 ms (client timeout 1000 ms). Trace:
W20260808 13:35:27.093519 4832 rpcz_store.cc:269] 0808 13:35:23.419810 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.541078 (+121268us) service_pool.cc:227] Handling call
0808 13:35:23.626554 (+ 85476us) spinlock_profiling.cc:243] Waited 48.1 ms on lock 0x7b5400003110. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227511ef11 00007f22752a3320 00007f227e0f8c46 00007f227e1fe4fe 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:27.092968 (+3466414us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25177,"reported_tablets":24,"spinlock_wait_cycles":110729984}
I20260808 13:35:27.099141 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 2173e97590be43f9a24f55689d796269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.107622 5609 tablet_service.cc:1467] Processing CreateTablet for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f (DEFAULT_TABLE table=default.Table_aa07254565ac49d4aadd887511d6838b [id=14177aacb9124b618a01b8e2ff32abb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.109130 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.114344 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.124795 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a26598a69146e1950e9227523fff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.125730 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 973a7afdcbf944aab8f11b3fd3995865 with delete_type TABLET_DATA_DELETED (replaced by tablet 3424ca753bcc4f448d314c56aba3323e) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.126506 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e with delete_type TABLET_DATA_DELETED (replaced by tablet ba39706cff65404485959865fff1772a) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.128456 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet e992a0294c404cfb9541b57782a05fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.128455 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:27.132573 4841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 29) took 1275 ms (client timeout 1000 ms). Trace:
W20260808 13:35:27.133169 4839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 28) took 3652 ms (client timeout 1000 ms). Trace:
W20260808 13:35:27.135708 4839 rpcz_store.cc:269] 0808 13:35:23.480711 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.553898 (+ 73187us) service_pool.cc:227] Handling call
0808 13:35:27.133099 (+3579201us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":64884,"reported_tablets":21}
I20260808 13:35:27.132819 7014 ts_tablet_manager.cc:1918] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:27.138206 7014 log.cc:1196] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/9260370cf8cb46a7b699581e4e8d46db
W20260808 13:35:27.138818 4837 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 36) took 4809 ms (client timeout 1000 ms). Trace:
I20260808 13:35:27.139446 7014 ts_tablet_manager.cc:1939] T 9260370cf8cb46a7b699581e4e8d46db P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:27.142976 4839 master_service.cc:946] ReplaceTablet: received request to replace tablet daa59a18ae624bc8975bee2c460895a7 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:27.141464 7032 leader_election.cc:288] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:27.146867 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:27.142078 4837 rpcz_store.cc:269] 0808 13:35:22.329712 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:22.338869 (+ 9157us) service_pool.cc:227] Handling call
0808 13:35:27.138747 (+4799878us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:23.551392 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:23.551842 (+ 450us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:23.551870 (+ 28us) write_op.cc:435] Acquired schema lock
0808 13:35:23.551885 (+ 15us) tablet.cc:660] Decoding operations
0808 13:35:23.552181 (+ 296us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:23.552218 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:23.552298 (+ 80us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:23.552481 (+ 183us) tablet.cc:699] Row locks acquired
0808 13:35:23.552497 (+ 16us) write_op.cc:260] PREPARE: finished
0808 13:35:23.552617 (+ 120us) write_op.cc:270] Start()
0808 13:35:23.552738 (+ 121us) write_op.cc:276] Timestamp: P: 1786196123552540 usec, L: 0
0808 13:35:23.552759 (+ 21us) op_driver.cc:331] REPLICATION: starting
0808 13:35:23.555965 (+ 3206us) log.cc:842] Serialized 527 byte log entry
0808 13:35:23.561049 (+ 5084us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:23.867826 (+306777us) op_driver.cc:444] REPLICATION: finished
0808 13:35:23.868082 (+ 256us) write_op.cc:301] APPLY: starting
0808 13:35:23.868143 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:23.868248 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:23.868270 (+ 22us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:23.868507 (+ 237us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:23.868665 (+ 158us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:23.868732 (+ 67us) write_op.cc:312] APPLY: finished
0808 13:35:23.868923 (+ 191us) log.cc:842] Serialized 23 byte log entry
0808 13:35:23.869084 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:23.869146 (+ 62us) write_op.cc:454] Released schema lock
0808 13:35:23.869175 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:23.869376 (+ 201us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:23.869401 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5400,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":94,"apply.run_cpu_time_us":1478,"apply.run_wall_time_us":1477,"num_ops":1,"prepare.queue_time_us":3404,"prepare.run_cpu_time_us":5130,"prepare.run_wall_time_us":9891,"replication_time_us":314975,"spinlock_wait_cycles":5259648}]]}
I20260808 13:35:27.155714 7014 tablet_replica.cc:330] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:27.156545 7014 raft_consensus.cc:2229] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:27.157480 7014 raft_consensus.cc:2258] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:27.160265 7014 ts_tablet_manager.cc:1905] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:27.171535 7016 leader_election.cc:288] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:27.172714 6810 leader_election.cc:288] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:27.174505 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet b809b185ee9e4c658a050c41c9abc4ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.175212 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 543debff6da54d26bf62260bf8721ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.175359 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.175968 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.176566 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.178334 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e992a0294c404cfb9541b57782a05fc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:27.179275 5644 raft_consensus.cc:3037] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:27.183454 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3682c9c656c4556bc13ae3aeddd7c5e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:27.152380 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_2df92438979447cc89b7a5301a13d6a9" } modify_external_catalogs: true
I20260808 13:35:27.153926 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 9260370cf8cb46a7b699581e4e8d46db (table default.Table_4026b16c74ac4327ac4ba5f178bfcaf6 [id=b33430fc55db40dd8eb1c9e0396a093b]) successfully deleted
W20260808 13:35:27.191994 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet f0c76b819474468baac51365360becfc because the tablet was not found. No further retry: Not found: Tablet not found: f0c76b819474468baac51365360becfc
W20260808 13:35:27.192646 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet b809b185ee9e4c658a050c41c9abc4ad because the tablet was not found. No further retry: Not found: Tablet not found: b809b185ee9e4c658a050c41c9abc4ad
W20260808 13:35:27.193254 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 2173e97590be43f9a24f55689d796269 because the tablet was not found. No further retry: Not found: Tablet not found: 2173e97590be43f9a24f55689d796269
I20260808 13:35:27.194706 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_2df92438979447cc89b7a5301a13d6a9" } modify_external_catalogs: true
W20260808 13:35:27.159185 4836 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 39) took 1599 ms (client timeout 1000 ms). Trace:
W20260808 13:35:27.199339 4836 rpcz_store.cc:269] 0808 13:35:25.559837 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:25.594532 (+ 34695us) service_pool.cc:227] Handling call
0808 13:35:25.700504 (+105972us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:27.159111 (+1458607us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":95479,"reported_tablets":24,"spinlock_wait_cycles":230433664}
I20260808 13:35:27.207652 4836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5c71e9adaaf5477b996f88813566c561" } modify_external_catalogs: true
I20260808 13:35:27.212625 7192 tablet.cc:1722] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:27.187990 5551 leader_election.cc:302] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:27.236812 7032 raft_consensus.cc:2781] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:27.237171 7032 raft_consensus.cc:482] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:27.237483 7032 raft_consensus.cc:3037] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:27.238682 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a26598a69146e1950e9227523fff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.239329 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.239923 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.240978 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 3424ca753bcc4f448d314c56aba3323e (DEFAULT_TABLE table=default.Table_5c71e9adaaf5477b996f88813566c561 [id=eb20921cee2f48dc86300ce8ce5f52a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.241989 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3424ca753bcc4f448d314c56aba3323e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.243264 5845 tablet_service.cc:1467] Processing CreateTablet for tablet b0f778e7fa51414cb209fbe1aea92739 (DEFAULT_TABLE table=default.Table_d56fc9f7ea3a43f98fe5a3057b51dbac [id=60103bc8b4444b688fdf2baab858ebf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.244221 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0f778e7fa51414cb209fbe1aea92739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.245093 7014 ts_tablet_manager.cc:1918] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:27.245481 7014 log.cc:1196] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/06c59eea10ec45af8e3a0992f9cc74bf
I20260808 13:35:27.246421 7014 ts_tablet_manager.cc:1939] T 06c59eea10ec45af8e3a0992f9cc74bf P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
W20260808 13:35:27.247352 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 59508.8us
I20260808 13:35:27.216856 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6826e4e4db2043b3a4fe9e7a9359cab6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:27.248958 5877 raft_consensus.cc:2454] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:27.198828 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet a541ec35b1a44109866fabc953f14cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.203758 6026 tablet_bootstrap.cc:492] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:27.251011 6026 ts_tablet_manager.cc:1397] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.407s user 0.006s sys 0.008s
I20260808 13:35:27.253376 6026 raft_consensus.cc:348] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.254045 6026 raft_consensus.cc:374] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:27.254285 6026 raft_consensus.cc:729] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:27.220513 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 3424ca753bcc4f448d314c56aba3323e (DEFAULT_TABLE table=default.Table_5c71e9adaaf5477b996f88813566c561 [id=eb20921cee2f48dc86300ce8ce5f52a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.254640 7014 tablet_replica.cc:330] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:27.255282 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3424ca753bcc4f448d314c56aba3323e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.223994 5606 tablet_service.cc:1467] Processing CreateTablet for tablet b0f778e7fa51414cb209fbe1aea92739 (DEFAULT_TABLE table=default.Table_d56fc9f7ea3a43f98fe5a3057b51dbac [id=60103bc8b4444b688fdf2baab858ebf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.259325 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0f778e7fa51414cb209fbe1aea92739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.269726 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 2173e97590be43f9a24f55689d796269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.271064 6026 consensus_queue.cc:260] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.278999 6026 ts_tablet_manager.cc:1428] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20260808 13:35:27.229835 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 973a7afdcbf944aab8f11b3fd3995865 with delete_type TABLET_DATA_DELETED (replaced by tablet 3424ca753bcc4f448d314c56aba3323e) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.280658 7199 tablet_replica.cc:330] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:27.281538 7199 raft_consensus.cc:2229] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:27.282414 7199 raft_consensus.cc:2258] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:27.283800 6026 tablet_bootstrap.cc:492] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:27.245641 7032 raft_consensus.cc:504] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.300369 5636 raft_consensus.cc:3037] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:27.303247 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6826e4e4db2043b3a4fe9e7a9359cab6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:27.303838 5877 raft_consensus.cc:3037] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:27.311273 7199 ts_tablet_manager.cc:1905] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:27.323005 5877 raft_consensus.cc:2454] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:27.324124 5550 leader_election.cc:302] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:27.334399 6901 raft_consensus.cc:2781] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
W20260808 13:35:27.345894 5771 leader_election.cc:341] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:27.346580 5771 leader_election.cc:341] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:27.351056 6901 raft_consensus.cc:686] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:27.351989 6901 consensus_queue.cc:237] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.204264 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6c19755f3f9149e7bdac49809abbe2a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:27.132962 4841 rpcz_store.cc:269] 0808 13:35:25.856820 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:25.894454 (+ 37634us) service_pool.cc:227] Handling call
0808 13:35:25.971467 (+ 77013us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:27.132496 (+1161029us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:26.490932 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:26.491684 (+ 752us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:26.491749 (+ 65us) write_op.cc:435] Acquired schema lock
0808 13:35:26.491775 (+ 26us) tablet.cc:660] Decoding operations
0808 13:35:26.492223 (+ 448us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:26.492284 (+ 61us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:26.492317 (+ 33us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:26.492616 (+ 299us) tablet.cc:699] Row locks acquired
0808 13:35:26.492640 (+ 24us) write_op.cc:260] PREPARE: finished
0808 13:35:26.492871 (+ 231us) write_op.cc:270] Start()
0808 13:35:26.493018 (+ 147us) write_op.cc:276] Timestamp: P: 1786196126492751 usec, L: 0
0808 13:35:26.493042 (+ 24us) op_driver.cc:331] REPLICATION: starting
0808 13:35:26.493940 (+ 898us) log.cc:842] Serialized 527 byte log entry
0808 13:35:26.515407 (+ 21467us) op_driver.cc:444] REPLICATION: finished
0808 13:35:26.516031 (+ 624us) write_op.cc:301] APPLY: starting
0808 13:35:26.516101 (+ 70us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:26.516281 (+ 180us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:26.516446 (+ 165us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:26.516736 (+ 290us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:26.516955 (+ 219us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:26.517015 (+ 60us) write_op.cc:312] APPLY: finished
0808 13:35:26.517243 (+ 228us) log.cc:842] Serialized 23 byte log entry
0808 13:35:26.517429 (+ 186us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:26.517489 (+ 60us) write_op.cc:454] Released schema lock
0808 13:35:26.517563 (+ 74us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:26.517867 (+ 304us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:26.517912 (+ 45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35552,"reported_tablets":21,"spinlock_wait_cycles":25112832,"child_traces":[["op",{"apply.queue_time_us":387,"apply.run_cpu_time_us":2086,"apply.run_wall_time_us":2085,"num_ops":1,"prepare.queue_time_us":1585,"prepare.run_cpu_time_us":3539,"prepare.run_wall_time_us":5017,"raft.queue_time_us":281,"raft.run_cpu_time_us":1009,"raft.run_wall_time_us":1010,"replication_time_us":22232}]]}
I20260808 13:35:27.249346 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 06c59eea10ec45af8e3a0992f9cc74bf (table default.Table_0c2610996a984e98b9d1899f23a40bf5 [id=d158bea8f5fc445e98a01e20b8a53651]) successfully deleted
W20260808 13:35:27.357342 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c19755f3f9149e7bdac49809abbe2a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:27.357354 4821 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 108029us
W20260808 13:35:27.286353 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet b809b185ee9e4c658a050c41c9abc4ad because the tablet was not found. No further retry: Not found: Tablet not found: b809b185ee9e4c658a050c41c9abc4ad
W20260808 13:35:27.359000 4819 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 72679.1us
W20260808 13:35:27.359524 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 543debff6da54d26bf62260bf8721ff7 because the tablet was not found. No further retry: Not found: Tablet not found: 543debff6da54d26bf62260bf8721ff7
W20260808 13:35:27.344509 5284 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 124044us
I20260808 13:35:27.362020 7014 raft_consensus.cc:2229] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:27.362515 7014 raft_consensus.cc:2258] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:27.363842 6033 tablet_bootstrap.cc:492] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:27.364267 6033 ts_tablet_manager.cc:1397] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.526s user 0.007s sys 0.006s
I20260808 13:35:27.364279 7014 ts_tablet_manager.cc:1905] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:27.363739 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_33484efb632a4860bc2485f6713c84d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:27.367650 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33484efb632a4860bc2485f6713c84d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:27.369942 7032 leader_election.cc:288] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:27.377019 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_7a1963a007954f8ca1d330f9d1936dd3 [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 3 got 2
I20260808 13:35:27.378099 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:27.378785 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:27.379346 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 1947f03e107447bb8e23fe5deaf1ea60 (table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a]). Expected version 2 got 0
I20260808 13:35:27.380164 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:27.380806 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet a8e18e51c2e24d3caa3d29e6645e46ae (table default.Table_321b4eb3dd024925af1cb354d3fe1567 [id=3c108c9e89424d9a9e11554601cf427c]). Expected version 1 got 0
I20260808 13:35:27.381577 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 4625b632d67244fc86e48d739dde707e (table default.Table_1665f65d290446e79ac5f6550ec733b7 [id=2f47077a0fd6488ebc33ab8794c0a5b6]). Expected version 1 got 0
I20260808 13:35:27.382345 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:27.390095 7014 ts_tablet_manager.cc:1918] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:27.390640 7014 log.cc:1196] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/45a26598a69146e1950e9227523fff13
I20260808 13:35:27.391449 7014 ts_tablet_manager.cc:1939] T 45a26598a69146e1950e9227523fff13 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:27.394212 7014 tablet_replica.cc:330] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:27.400132 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet daa59a18ae624bc8975bee2c460895a7
I20260808 13:35:27.401412 4836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5c71e9adaaf5477b996f88813566c561" } modify_external_catalogs: true
I20260808 13:35:27.410694 7199 ts_tablet_manager.cc:1918] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:27.411264 7199 log.cc:1196] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/f0c76b819474468baac51365360becfc
I20260808 13:35:27.413178 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet daa59a18ae624bc8975bee2c460895a7 with delete_type TABLET_DATA_DELETED (replaced by tablet cb7f6487e28f4aaa94d448be71f8a3c2) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.414847 7199 ts_tablet_manager.cc:1939] T f0c76b819474468baac51365360becfc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:27.419131 7199 tablet_replica.cc:330] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:27.419271 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet f0c76b819474468baac51365360becfc (table default.Table_c4e53d8c393449429ef2791497ee7208 [id=26fb789fff434bb0825daf68d5378013]) successfully deleted
I20260808 13:35:27.420035 7199 raft_consensus.cc:2229] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:27.413760 4839 catalog_manager.cc:6565] ReplaceTablet: tablet daa59a18ae624bc8975bee2c460895a7 deleted and replaced by tablet cb7f6487e28f4aaa94d448be71f8a3c2
I20260808 13:35:27.424249 4839 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60270 (request call id 282) took 2124 ms. Trace:
I20260808 13:35:27.420629 7199 raft_consensus.cc:2258] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:27.429579 7199 ts_tablet_manager.cc:1905] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:27.436317 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:27.437350 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_7a1963a007954f8ca1d330f9d1936dd3 [id=d5b4528cac46486e857db6176b4b3a2b]). Expected version 3 got 2
I20260808 13:35:27.441881 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet b314734698ed423ba9cf4134db5846ec
I20260808 13:35:27.434793 6033 raft_consensus.cc:348] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.442961 6033 raft_consensus.cc:374] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:27.443333 6033 raft_consensus.cc:729] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:27.443941 6033 consensus_queue.cc:260] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.449157 6033 ts_tablet_manager.cc:1428] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.085s user 0.006s sys 0.002s
I20260808 13:35:27.450039 6033 tablet_bootstrap.cc:492] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:27.451825 6046 tablet_bootstrap.cc:654] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:27.456868 5345 tablet_service.cc:1467] Processing CreateTablet for tablet 3424ca753bcc4f448d314c56aba3323e (DEFAULT_TABLE table=default.Table_5c71e9adaaf5477b996f88813566c561 [id=eb20921cee2f48dc86300ce8ce5f52a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.457887 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3424ca753bcc4f448d314c56aba3323e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.458461 5346 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b29786cb404bc2b01473c3b8390a50 with delete_type TABLET_DATA_DELETED (replaced by tablet 362dbd6990e740b39690537af98aa2f8) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.458930 5353 tablet_service.cc:1514] Processing DeleteTablet for tablet daa59a18ae624bc8975bee2c460895a7 with delete_type TABLET_DATA_DELETED (replaced by tablet cb7f6487e28f4aaa94d448be71f8a3c2) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.459913 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fb3e099f7f4544b3c100070f0d14ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.460208 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet b809b185ee9e4c658a050c41c9abc4ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.461815 4842 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 283) took 1727 ms. Trace:
I20260808 13:35:27.466190 4841 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 286) took 1573 ms. Trace:
I20260808 13:35:27.468775 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet b314734698ed423ba9cf4134db5846ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:27 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.468801 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet 543debff6da54d26bf62260bf8721ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:09 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.470817 5356 tablet_service.cc:1467] Processing CreateTablet for tablet b0f778e7fa51414cb209fbe1aea92739 (DEFAULT_TABLE table=default.Table_d56fc9f7ea3a43f98fe5a3057b51dbac [id=60103bc8b4444b688fdf2baab858ebf5]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:27.476250 4819 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 56995.5us
W20260808 13:35:27.477582 4833 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 44) took 4519 ms (client timeout 1000 ms). Trace:
I20260808 13:35:27.478186 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet b314734698ed423ba9cf4134db5846ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:27 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:27.468515 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:27.477927 4833 rpcz_store.cc:269] 0808 13:35:22.958575 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.532010 (+573435us) service_pool.cc:227] Handling call
0808 13:35:27.477512 (+3945502us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:24.437303 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:24.437725 (+ 422us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:24.437762 (+ 37us) write_op.cc:435] Acquired schema lock
0808 13:35:24.437777 (+ 15us) tablet.cc:660] Decoding operations
0808 13:35:24.438068 (+ 291us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:24.438115 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:24.438143 (+ 28us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:24.438302 (+ 159us) tablet.cc:699] Row locks acquired
0808 13:35:24.438316 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:24.438433 (+ 117us) write_op.cc:270] Start()
0808 13:35:24.438518 (+ 85us) write_op.cc:276] Timestamp: P: 1786196124438362 usec, L: 0
0808 13:35:24.438536 (+ 18us) op_driver.cc:331] REPLICATION: starting
0808 13:35:24.451151 (+ 12615us) log.cc:842] Serialized 356 byte log entry
0808 13:35:24.535608 (+ 84457us) op_driver.cc:444] REPLICATION: finished
0808 13:35:24.560502 (+ 24894us) write_op.cc:301] APPLY: starting
0808 13:35:24.560563 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:24.560662 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:24.560780 (+ 118us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:24.561050 (+ 270us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:24.561189 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:24.561246 (+ 57us) write_op.cc:312] APPLY: finished
0808 13:35:24.561443 (+ 197us) log.cc:842] Serialized 23 byte log entry
0808 13:35:24.561589 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.561650 (+ 61us) write_op.cc:454] Released schema lock
0808 13:35:24.561677 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:24.561858 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:24.561884 (+ 26us) write_op.cc:454] Released schema lock
Related trace 'op':
0808 13:35:26.498755 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:26.499583 (+ 828us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:26.499623 (+ 40us) write_op.cc:435] Acquired schema lock
0808 13:35:26.499658 (+ 35us) tablet.cc:660] Decoding operations
0808 13:35:26.500256 (+ 598us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:26.500364 (+ 108us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:26.500421 (+ 57us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:26.500776 (+ 355us) tablet.cc:699] Row locks acquired
0808 13:35:26.500805 (+ 29us) write_op.cc:260] PREPARE: finished
0808 13:35:26.501140 (+ 335us) write_op.cc:270] Start()
0808 13:35:26.501322 (+ 182us) write_op.cc:276] Timestamp: P: 1786196126500988 usec, L: 0
0808 13:35:26.501392 (+ 70us) op_driver.cc:331] REPLICATION: starting
0808 13:35:26.503640 (+ 2248us) log.cc:842] Serialized 356 byte log entry
0808 13:35:26.524557 (+ 20917us) op_driver.cc:444] REPLICATION: finished
0808 13:35:26.525925 (+ 1368us) write_op.cc:301] APPLY: starting
0808 13:35:26.525978 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:26.526069 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:26.526192 (+ 123us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:26.526393 (+ 201us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:26.526532 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:26.526572 (+ 40us) write_op.cc:312] APPLY: finished
0808 13:35:26.526853 (+ 281us) log.cc:842] Serialized 23 byte log entry
0808 13:35:26.526996 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:26.527062 (+ 66us) write_op.cc:454] Released schema lock
0808 13:35:26.527090 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:26.527249 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:26.527285 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":284040,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":24721,"apply.run_cpu_time_us":1507,"apply.run_wall_time_us":1557,"num_ops":1,"prepare.queue_time_us":488643,"prepare.run_cpu_time_us":2066,"prepare.run_wall_time_us":14068,"replication_time_us":96993,"thread_start_us":25900,"threads_started":1}],["op",{"apply.queue_time_us":1197,"apply.run_cpu_time_us":1453,"apply.run_wall_time_us":1515,"num_ops":1,"prepare.queue_time_us":1533,"prepare.run_cpu_time_us":4096,"prepare.run_wall_time_us":5687,"raft.queue_time_us":109,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":12,"replication_time_us":23093}]]}
I20260808 13:35:27.461124 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a26598a69146e1950e9227523fff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.469734 7013 ts_tablet_manager.cc:1918] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:27.425267 4839 rpcz_store.cc:276] 0808 13:35:25.299714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.142849 (+1843135us) service_pool.cc:227] Handling call
0808 13:35:27.424020 (+281171us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.161686 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:27.162519 (+ 833us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:27.162548 (+ 29us) write_op.cc:435] Acquired schema lock
0808 13:35:27.162574 (+ 26us) tablet.cc:660] Decoding operations
0808 13:35:27.163144 (+ 570us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:27.163239 (+ 95us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:27.163300 (+ 61us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:27.163631 (+ 331us) tablet.cc:699] Row locks acquired
0808 13:35:27.163678 (+ 47us) write_op.cc:260] PREPARE: finished
0808 13:35:27.163868 (+ 190us) write_op.cc:270] Start()
0808 13:35:27.164036 (+ 168us) write_op.cc:276] Timestamp: P: 1786196127163786 usec, L: 0
0808 13:35:27.164053 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.164881 (+ 828us) log.cc:842] Serialized 622 byte log entry
0808 13:35:27.312380 (+147499us) op_driver.cc:444] REPLICATION: finished
0808 13:35:27.312718 (+ 338us) write_op.cc:301] APPLY: starting
0808 13:35:27.312777 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:27.312913 (+ 136us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:27.313016 (+ 103us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:27.313400 (+ 384us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:27.313641 (+ 241us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:27.313689 (+ 48us) write_op.cc:312] APPLY: finished
0808 13:35:27.313898 (+ 209us) log.cc:842] Serialized 29 byte log entry
0808 13:35:27.314784 (+ 886us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.314861 (+ 77us) write_op.cc:454] Released schema lock
0808 13:35:27.314897 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:27.316414 (+ 1517us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.316447 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":165,"apply.run_cpu_time_us":2085,"apply.run_wall_time_us":4093,"num_ops":2,"prepare.queue_time_us":113,"prepare.run_cpu_time_us":3740,"prepare.run_wall_time_us":7680,"raft.queue_time_us":125,"raft.run_cpu_time_us":1043,"raft.run_wall_time_us":1043,"replication_time_us":148216}]]}
I20260808 13:35:27.491430 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0f778e7fa51414cb209fbe1aea92739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.503265 6026 tablet_bootstrap.cc:654] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:27.492439 4838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 40) took 787 ms (client timeout 1000 ms). Trace:
W20260808 13:35:27.506198 4838 rpcz_store.cc:269] 0808 13:35:26.705098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.053046 (+347948us) service_pool.cc:227] Handling call
0808 13:35:27.492275 (+439229us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":56076,"reported_tablets":25}
I20260808 13:35:27.507270 4838 master_service.cc:946] ReplaceTablet: received request to replace tablet 2173e97590be43f9a24f55689d796269 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:27.506947 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_52f5a796f61f4be79a196bc23065491b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:27.508451 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52f5a796f61f4be79a196bc23065491b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:27.462186 4842 rpcz_store.cc:276] 0808 13:35:25.734839 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.152040 (+1417201us) service_pool.cc:227] Handling call
0808 13:35:27.195265 (+ 43225us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:27.195946 (+ 681us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:27.196085 (+ 139us) catalog_manager.cc:2761] Locking tablets
0808 13:35:27.196684 (+ 599us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:27.441469 (+244785us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:27.441692 (+ 223us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:27.441797 (+ 105us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:27.461751 (+ 19954us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.402720 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:27.403165 (+ 445us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:27.403191 (+ 26us) write_op.cc:435] Acquired schema lock
0808 13:35:27.403208 (+ 17us) tablet.cc:660] Decoding operations
0808 13:35:27.403596 (+ 388us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:27.403644 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:27.403673 (+ 29us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:27.404022 (+ 349us) tablet.cc:699] Row locks acquired
0808 13:35:27.404045 (+ 23us) write_op.cc:260] PREPARE: finished
0808 13:35:27.404181 (+ 136us) write_op.cc:270] Start()
0808 13:35:27.404285 (+ 104us) write_op.cc:276] Timestamp: P: 1786196127404109 usec, L: 0
0808 13:35:27.404306 (+ 21us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.405298 (+ 992us) log.cc:842] Serialized 725 byte log entry
0808 13:35:27.419065 (+ 13767us) op_driver.cc:444] REPLICATION: finished
0808 13:35:27.419504 (+ 439us) write_op.cc:301] APPLY: starting
0808 13:35:27.419557 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:27.419664 (+ 107us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:27.419758 (+ 94us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:27.420020 (+ 262us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:27.420155 (+ 135us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:27.420206 (+ 51us) write_op.cc:312] APPLY: finished
0808 13:35:27.420508 (+ 302us) log.cc:842] Serialized 29 byte log entry
0808 13:35:27.420654 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.420724 (+ 70us) write_op.cc:454] Released schema lock
0808 13:35:27.420761 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:27.420940 (+ 179us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.420967 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":290,"apply.run_cpu_time_us":1667,"apply.run_wall_time_us":1666,"num_ops":2,"prepare.queue_time_us":7392,"prepare.run_cpu_time_us":3112,"prepare.run_wall_time_us":25128,"raft.queue_time_us":5580,"raft.run_cpu_time_us":1079,"raft.run_wall_time_us":1080,"replication_time_us":14687}]]}
I20260808 13:35:27.510820 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3" } new_table_name: "default.Table_9d2867cdebf84440b7aa21a71faba9c4" modify_external_catalogs: true
I20260808 13:35:27.511018 7039 consensus_queue.cc:1035] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:27.466581 4841 rpcz_store.cc:276] 0808 13:35:25.893179 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.361998 (+1468819us) service_pool.cc:227] Handling call
0808 13:35:27.401964 (+ 39966us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:27.403582 (+ 1618us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:27.465598 (+ 62016us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:27.465890 (+ 292us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:27.466130 (+ 240us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.431259 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:27.432138 (+ 879us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:27.432188 (+ 50us) write_op.cc:435] Acquired schema lock
0808 13:35:27.432200 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:27.432574 (+ 374us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:27.432621 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:27.432645 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:27.432967 (+ 322us) tablet.cc:699] Row locks acquired
0808 13:35:27.432982 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:27.433166 (+ 184us) write_op.cc:270] Start()
0808 13:35:27.433371 (+ 205us) write_op.cc:276] Timestamp: P: 1786196127433080 usec, L: 0
0808 13:35:27.433388 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.434101 (+ 713us) log.cc:842] Serialized 433 byte log entry
0808 13:35:27.445775 (+ 11674us) op_driver.cc:444] REPLICATION: finished
0808 13:35:27.446175 (+ 400us) write_op.cc:301] APPLY: starting
0808 13:35:27.446228 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:27.446374 (+ 146us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:27.446477 (+ 103us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:27.447104 (+ 627us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:27.447310 (+ 206us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:27.447388 (+ 78us) write_op.cc:312] APPLY: finished
0808 13:35:27.447609 (+ 221us) log.cc:842] Serialized 29 byte log entry
0808 13:35:27.448075 (+ 466us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.448208 (+ 133us) write_op.cc:454] Released schema lock
0808 13:35:27.448265 (+ 57us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:27.448603 (+ 338us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.448648 (+ 45us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":203,"apply.run_cpu_time_us":2512,"apply.run_wall_time_us":2682,"num_ops":2,"prepare.queue_time_us":347,"prepare.run_cpu_time_us":3187,"prepare.run_wall_time_us":4951,"raft.queue_time_us":120,"raft.run_cpu_time_us":649,"raft.run_wall_time_us":650,"replication_time_us":12311}]]}
I20260808 13:35:27.512523 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_17e6c38083384a1b94be67b7a240e85b" } new_table_name: "default.Table_95ed126fb41342c1a6a1539f0a5ebbe3" modify_external_catalogs: true
W20260808 13:35:27.494467 4840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 45) took 2799 ms (client timeout 1000 ms). Trace:
I20260808 13:35:27.494714 7013 log.cc:1196] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/2173e97590be43f9a24f55689d796269
I20260808 13:35:27.515357 7013 ts_tablet_manager.cc:1939] T 2173e97590be43f9a24f55689d796269 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
W20260808 13:35:27.513968 4840 rpcz_store.cc:269] 0808 13:35:24.695131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.772919 (+ 77788us) service_pool.cc:227] Handling call
0808 13:35:24.806163 (+ 33244us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:27.494400 (+2688237us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:25.709627 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:25.710924 (+ 1297us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:25.710977 (+ 53us) write_op.cc:435] Acquired schema lock
0808 13:35:25.710999 (+ 22us) tablet.cc:660] Decoding operations
0808 13:35:25.711603 (+ 604us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:25.711677 (+ 74us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:25.711704 (+ 27us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:25.712043 (+ 339us) tablet.cc:699] Row locks acquired
0808 13:35:25.712089 (+ 46us) write_op.cc:260] PREPARE: finished
0808 13:35:25.712363 (+ 274us) write_op.cc:270] Start()
0808 13:35:25.712552 (+ 189us) write_op.cc:276] Timestamp: P: 1786196125712272 usec, L: 0
0808 13:35:25.712597 (+ 45us) op_driver.cc:331] REPLICATION: starting
0808 13:35:25.713712 (+ 1115us) log.cc:842] Serialized 527 byte log entry
0808 13:35:25.760928 (+ 47216us) op_driver.cc:444] REPLICATION: finished
0808 13:35:25.764930 (+ 4002us) write_op.cc:301] APPLY: starting
0808 13:35:25.765033 (+ 103us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:25.765241 (+ 208us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:25.765372 (+ 131us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:25.765591 (+ 219us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:25.765835 (+ 244us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:25.765976 (+ 141us) write_op.cc:312] APPLY: finished
0808 13:35:25.766311 (+ 335us) log.cc:842] Serialized 23 byte log entry
0808 13:35:25.766579 (+ 268us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:25.767888 (+ 1309us) write_op.cc:454] Released schema lock
0808 13:35:25.767949 (+ 61us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:25.768305 (+ 356us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:25.768351 (+ 46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":100658,"reported_tablets":24,"spinlock_wait_cycles":48063744,"child_traces":[["op",{"apply.queue_time_us":3731,"apply.run_cpu_time_us":2563,"apply.run_wall_time_us":3709,"num_ops":1,"prepare.queue_time_us":80261,"prepare.run_cpu_time_us":4671,"prepare.run_wall_time_us":109142,"raft.queue_time_us":147,"raft.run_cpu_time_us":1185,"raft.run_wall_time_us":1184,"replication_time_us":48216}]]}
I20260808 13:35:27.438395 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:27.517477 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 210) took 3865 ms. Trace:
I20260808 13:35:27.518016 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:27.517767 7013 rpcz_store.cc:276] 0808 13:35:23.652262 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.652380 (+ 118us) service_pool.cc:227] Handling call
0808 13:35:26.458949 (+2806569us) ts_tablet_manager.cc:1145] Deleting tablet 2173e97590be43f9a24f55689d796269
0808 13:35:26.458998 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:26.890874 (+431876us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:27.439627 (+548753us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:27.517419 (+ 77792us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2805990}
W20260808 13:35:27.518805 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 46) took 1287 ms (client timeout 1000 ms). Trace:
I20260808 13:35:27.519507 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 1947f03e107447bb8e23fe5deaf1ea60 (table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a]). Expected version 2 got 0
I20260808 13:35:27.520267 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet a8e18e51c2e24d3caa3d29e6645e46ae (table default.Table_321b4eb3dd024925af1cb354d3fe1567 [id=3c108c9e89424d9a9e11554601cf427c]). Expected version 1 got 0
W20260808 13:35:27.519105 4835 rpcz_store.cc:269] 0808 13:35:26.231580 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:26.452072 (+220492us) service_pool.cc:227] Handling call
0808 13:35:27.518737 (+1066665us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.058012 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:27.058684 (+ 672us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:27.058738 (+ 54us) write_op.cc:435] Acquired schema lock
0808 13:35:27.058751 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:27.059889 (+ 1138us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:27.059963 (+ 74us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:27.060053 (+ 90us) tablet.cc:683] Acquiring locks for 3 operations
0808 13:35:27.060717 (+ 664us) tablet.cc:699] Row locks acquired
0808 13:35:27.060746 (+ 29us) write_op.cc:260] PREPARE: finished
0808 13:35:27.060936 (+ 190us) write_op.cc:270] Start()
0808 13:35:27.061087 (+ 151us) write_op.cc:276] Timestamp: P: 1786196127060835 usec, L: 0
0808 13:35:27.061118 (+ 31us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.062090 (+ 972us) log.cc:842] Serialized 1275 byte log entry
0808 13:35:27.107117 (+ 45027us) op_driver.cc:444] REPLICATION: finished
0808 13:35:27.107631 (+ 514us) write_op.cc:301] APPLY: starting
0808 13:35:27.107709 (+ 78us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:27.107949 (+ 240us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:27.108114 (+ 165us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:27.108691 (+ 577us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:27.108949 (+ 258us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0808 13:35:27.109025 (+ 76us) write_op.cc:312] APPLY: finished
0808 13:35:27.109431 (+ 406us) log.cc:842] Serialized 35 byte log entry
0808 13:35:27.109671 (+ 240us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.109762 (+ 91us) write_op.cc:454] Released schema lock
0808 13:35:27.109865 (+ 103us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:27.110112 (+ 247us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.110150 (+ 38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":86196,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":329,"apply.run_cpu_time_us":2782,"apply.run_wall_time_us":2782,"num_ops":3,"prepare.queue_time_us":4213,"prepare.run_cpu_time_us":4552,"prepare.run_wall_time_us":65033,"raft.queue_time_us":308,"raft.run_cpu_time_us":1170,"raft.run_wall_time_us":1170,"replication_time_us":45901}]]}
I20260808 13:35:27.523743 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a7a4e8c009fd4046ad8a17c30d9c4de5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:27.525943 4838 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60270 (request call id 288) took 1294 ms. Trace:
I20260808 13:35:27.526252 4838 rpcz_store.cc:276] 0808 13:35:26.231947 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.507147 (+1275200us) service_pool.cc:227] Handling call
0808 13:35:27.525884 (+ 18737us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260808 13:35:27.534443 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7a4e8c009fd4046ad8a17c30d9c4de5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:27.469899 4837 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 284) took 1705 ms. Trace:
I20260808 13:35:27.539924 4837 rpcz_store.cc:276] 0808 13:35:25.764207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.203660 (+1439453us) service_pool.cc:227] Handling call
0808 13:35:27.403812 (+200152us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:27.424717 (+ 20905us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:27.468302 (+ 43585us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:27.468838 (+ 536us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:27.469207 (+ 369us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.447044 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:27.448001 (+ 957us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:27.448062 (+ 61us) write_op.cc:435] Acquired schema lock
0808 13:35:27.448108 (+ 46us) tablet.cc:660] Decoding operations
0808 13:35:27.448682 (+ 574us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:27.448790 (+ 108us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:27.448856 (+ 66us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:27.449224 (+ 368us) tablet.cc:699] Row locks acquired
0808 13:35:27.449266 (+ 42us) write_op.cc:260] PREPARE: finished
0808 13:35:27.449425 (+ 159us) write_op.cc:270] Start()
0808 13:35:27.449562 (+ 137us) write_op.cc:276] Timestamp: P: 1786196127449333 usec, L: 0
0808 13:35:27.449605 (+ 43us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.464038 (+ 14433us) log.cc:842] Serialized 433 byte log entry
0808 13:35:27.465745 (+ 1707us) op_driver.cc:444] REPLICATION: finished
0808 13:35:27.466329 (+ 584us) write_op.cc:301] APPLY: starting
0808 13:35:27.466408 (+ 79us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:27.466567 (+ 159us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:27.466738 (+ 171us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:27.467110 (+ 372us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:27.467251 (+ 141us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:27.467291 (+ 40us) write_op.cc:312] APPLY: finished
0808 13:35:27.467454 (+ 163us) log.cc:842] Serialized 29 byte log entry
0808 13:35:27.467597 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.467674 (+ 77us) write_op.cc:454] Released schema lock
0808 13:35:27.467720 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:27.467895 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.467918 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":358,"apply.run_cpu_time_us":1764,"apply.run_wall_time_us":1764,"mutex_wait_us":3863,"num_ops":2,"prepare.queue_time_us":259,"prepare.run_cpu_time_us":3380,"prepare.run_wall_time_us":51125,"raft.queue_time_us":4696,"raft.run_cpu_time_us":521,"raft.run_wall_time_us":529,"replication_time_us":16023}]]}
I20260808 13:35:27.589470 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 211) took 3936 ms. Trace:
I20260808 13:35:27.591173 7013 rpcz_store.cc:276] 0808 13:35:23.673675 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:23.767527 (+ 93852us) service_pool.cc:227] Handling call
0808 13:35:27.589046 (+3821519us) ts_tablet_manager.cc:1145] Deleting tablet 196d80d806b0432cacf2ca6aa0da8c6b
0808 13:35:27.589104 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:27.589407 (+ 303us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3820959}
W20260808 13:35:27.593977 5334 consensus_meta.cc:287] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c: Time spent flushing consensus metadata: real 0.559s user 0.008s sys 0.000s
I20260808 13:35:27.596113 7199 ts_tablet_manager.cc:1918] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:27.596522 7199 log.cc:1196] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/06c59eea10ec45af8e3a0992f9cc74bf
I20260808 13:35:27.597369 7199 ts_tablet_manager.cc:1939] T 06c59eea10ec45af8e3a0992f9cc74bf P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:27.600294 7199 tablet_replica.cc:330] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:27.600769 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 06c59eea10ec45af8e3a0992f9cc74bf (table default.Table_0c2610996a984e98b9d1899f23a40bf5 [id=d158bea8f5fc445e98a01e20b8a53651]) successfully deleted
I20260808 13:35:27.601125 7199 raft_consensus.cc:2229] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 13:35:27.601475 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet f0c76b819474468baac51365360becfc because the tablet was not found. No further retry: Not found: Tablet not found: f0c76b819474468baac51365360becfc
I20260808 13:35:27.601641 7199 raft_consensus.cc:2258] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:27.603538 7199 ts_tablet_manager.cc:1905] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:27.600757 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 222) took 3141 ms. Trace:
I20260808 13:35:27.604018 7013 rpcz_store.cc:276] 0808 13:35:24.459242 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.459576 (+ 334us) service_pool.cc:227] Handling call
0808 13:35:27.597229 (+3137653us) ts_tablet_manager.cc:1145] Deleting tablet 86c2b34c95244472beec43643e60ff5f
0808 13:35:27.597267 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:27.597550 (+ 283us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3136865}
I20260808 13:35:27.605875 7013 tablet_replica.cc:330] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:27.613227 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 2173e97590be43f9a24f55689d796269 (table default.Table_d052232acdff41f394004ddc3252bbbd [id=6017dfb7d1574e88a452e2d539b1aa03]) successfully deleted
I20260808 13:35:27.611754 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6826e4e4db2043b3a4fe9e7a9359cab6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:27.632016 5386 raft_consensus.cc:3037] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:27.637290 7013 raft_consensus.cc:2229] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:27.638127 7013 raft_consensus.cc:2258] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:27.638242 5386 raft_consensus.cc:2454] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:27.640367 7013 ts_tablet_manager.cc:1905] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:27.650547 4839 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 287) took 1552 ms. Trace:
I20260808 13:35:27.657135 4839 rpcz_store.cc:276] 0808 13:35:26.098048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.506471 (+1408423us) service_pool.cc:227] Handling call
0808 13:35:27.526055 (+ 19584us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:27.529967 (+ 3912us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:27.649745 (+119778us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:27.650138 (+ 393us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:27.650467 (+ 329us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.632037 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:27.632562 (+ 525us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:27.632597 (+ 35us) write_op.cc:435] Acquired schema lock
0808 13:35:27.632622 (+ 25us) tablet.cc:660] Decoding operations
0808 13:35:27.633023 (+ 401us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:27.633082 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:27.633120 (+ 38us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:27.633456 (+ 336us) tablet.cc:699] Row locks acquired
0808 13:35:27.633478 (+ 22us) write_op.cc:260] PREPARE: finished
0808 13:35:27.633616 (+ 138us) write_op.cc:270] Start()
0808 13:35:27.633727 (+ 111us) write_op.cc:276] Timestamp: P: 1786196127633518 usec, L: 0
0808 13:35:27.633754 (+ 27us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.634427 (+ 673us) log.cc:842] Serialized 433 byte log entry
0808 13:35:27.646871 (+ 12444us) op_driver.cc:444] REPLICATION: finished
0808 13:35:27.647198 (+ 327us) write_op.cc:301] APPLY: starting
0808 13:35:27.647253 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:27.647367 (+ 114us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:27.647488 (+ 121us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:27.647974 (+ 486us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:27.648151 (+ 177us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:27.648192 (+ 41us) write_op.cc:312] APPLY: finished
0808 13:35:27.648390 (+ 198us) log.cc:842] Serialized 29 byte log entry
0808 13:35:27.648535 (+ 145us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.648612 (+ 77us) write_op.cc:454] Released schema lock
0808 13:35:27.648641 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:27.648810 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.648834 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":181,"apply.run_cpu_time_us":1780,"apply.run_wall_time_us":1779,"num_ops":2,"prepare.queue_time_us":50078,"raft.queue_time_us":212,"raft.run_cpu_time_us":675,"raft.run_wall_time_us":675,"replication_time_us":13036}]]}
W20260808 13:35:27.661164 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 196d80d806b0432cacf2ca6aa0da8c6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 196d80d806b0432cacf2ca6aa0da8c6b already in progress: creating tablet
I20260808 13:35:27.668462 7199 ts_tablet_manager.cc:1918] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:27.668821 7199 log.cc:1196] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/45a26598a69146e1950e9227523fff13
I20260808 13:35:27.669584 7199 ts_tablet_manager.cc:1939] T 45a26598a69146e1950e9227523fff13 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:27.684619 7199 tablet_replica.cc:330] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:27.684908 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 45a26598a69146e1950e9227523fff13 (table default.Table_909de5e2ab404a69ba774963323e2102 [id=e85188ee407c472cbab5ab0ed13d27e6]) successfully deleted
I20260808 13:35:27.685606 7199 raft_consensus.cc:2229] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:27.686383 7199 raft_consensus.cc:2258] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:27.688472 7199 ts_tablet_manager.cc:1905] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:27.695967 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 342027968b884f1cbc78b1371318dfc9 (DEFAULT_TABLE table=default.Table_6ea0a150e9a245babc374b7222ad1ca7 [id=c69587f42e4b437aadf52a706e37e070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.700112 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 4625b632d67244fc86e48d739dde707e (table default.Table_1665f65d290446e79ac5f6550ec733b7 [id=2f47077a0fd6488ebc33ab8794c0a5b6]). Expected version 1 got 0
I20260808 13:35:27.704737 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet a8e18e51c2e24d3caa3d29e6645e46ae (table default.Table_321b4eb3dd024925af1cb354d3fe1567 [id=3c108c9e89424d9a9e11554601cf427c]). Expected version 1 got 0
I20260808 13:35:27.708034 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 9f53e5b5c4bd4a9385ad198b283c6e79 (DEFAULT_TABLE table=default.Table_22ad421714294dac9da7cb4984aee8f3 [id=0cdf67cb0e1f453e950a53dd444920de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.708899 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:27.711041 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_aff1db9ef8c64a10bd5a402bf6b6d3d3 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 1 got 0
I20260808 13:35:27.712162 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 1947f03e107447bb8e23fe5deaf1ea60 (table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a]). Expected version 2 got 0
I20260808 13:35:27.715363 5832 tablet_service.cc:1467] Processing CreateTablet for tablet d2be108f26bb48d49216c6b7eef6b9b8 (DEFAULT_TABLE table=default.Table_cab0814763554410aa969d5b417710fb [id=eef293a0bbc94c2db94a64223ef71a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.718811 4833 catalog_manager.cc:5495] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:27.722265 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:27.721994 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 6e70a5cf88d94b36913a6d1ccfcb979a (DEFAULT_TABLE table=default.Table_8e189703ccf14f70903828148ce964df [id=8dd3a08c5ec440389be4746b139a39aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.728277 4832 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_1665f65d290446e79ac5f6550ec733b7 [id=2f47077a0fd6488ebc33ab8794c0a5b6] alter complete (version 1)
I20260808 13:35:27.729166 4842 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 289) took 1073 ms. Trace:
I20260808 13:35:27.729457 4842 rpcz_store.cc:276] 0808 13:35:26.655558 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.510476 (+854918us) service_pool.cc:227] Handling call
0808 13:35:27.527552 (+ 17076us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:27.528276 (+ 724us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:27.528917 (+ 641us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:27.529085 (+ 168us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:27.547563 (+ 18478us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:27.729097 (+181534us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.530314 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:27.530786 (+ 472us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:27.530816 (+ 30us) write_op.cc:435] Acquired schema lock
0808 13:35:27.530834 (+ 18us) tablet.cc:660] Decoding operations
0808 13:35:27.531138 (+ 304us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:27.531186 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:27.531212 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:27.531374 (+ 162us) tablet.cc:699] Row locks acquired
0808 13:35:27.531389 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:27.531506 (+ 117us) write_op.cc:270] Start()
0808 13:35:27.531609 (+ 103us) write_op.cc:276] Timestamp: P: 1786196127531426 usec, L: 0
0808 13:35:27.531631 (+ 22us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.532312 (+ 681us) log.cc:842] Serialized 405 byte log entry
0808 13:35:27.540300 (+ 7988us) op_driver.cc:444] REPLICATION: finished
0808 13:35:27.543499 (+ 3199us) write_op.cc:301] APPLY: starting
0808 13:35:27.543571 (+ 72us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:27.543683 (+ 112us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:27.543836 (+ 153us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:27.544029 (+ 193us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:27.544155 (+ 126us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:27.544197 (+ 42us) write_op.cc:312] APPLY: finished
0808 13:35:27.544369 (+ 172us) log.cc:842] Serialized 23 byte log entry
0808 13:35:27.544499 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.544561 (+ 62us) write_op.cc:454] Released schema lock
0808 13:35:27.544590 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:27.544761 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.544784 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3023,"apply.run_cpu_time_us":1436,"apply.run_wall_time_us":1436,"num_ops":1,"prepare.queue_time_us":116,"prepare.run_cpu_time_us":2305,"prepare.run_wall_time_us":2306,"raft.queue_time_us":164,"raft.run_cpu_time_us":672,"raft.run_wall_time_us":15093,"replication_time_us":8573}]]}
I20260808 13:35:27.745360 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f53e5b5c4bd4a9385ad198b283c6e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.756992 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e70a5cf88d94b36913a6d1ccfcb979a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.758948 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342027968b884f1cbc78b1371318dfc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.721942 6046 tablet_bootstrap.cc:492] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:27.722469 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.723537 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet 9260370cf8cb46a7b699581e4e8d46db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.771885 6046 ts_tablet_manager.cc:1397] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.821s user 0.012s sys 0.008s
I20260808 13:35:27.726451 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.759625 7219 consensus_queue.cc:1035] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:27.761437 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet a541ec35b1a44109866fabc953f14cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.762506 5345 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.780537 5832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2be108f26bb48d49216c6b7eef6b9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.782302 7199 ts_tablet_manager.cc:1918] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:27.788138 6046 raft_consensus.cc:348] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.788972 6046 raft_consensus.cc:374] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:27.789211 6046 raft_consensus.cc:729] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:27.789963 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.791769 5336 tablet_service.cc:1467] Processing CreateTablet for tablet 342027968b884f1cbc78b1371318dfc9 (DEFAULT_TABLE table=default.Table_6ea0a150e9a245babc374b7222ad1ca7 [id=c69587f42e4b437aadf52a706e37e070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.792927 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342027968b884f1cbc78b1371318dfc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.812634 7192 tablet.cc:1722] T 1947f03e107447bb8e23fe5deaf1ea60 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:27.818733 5599 tablet_service.cc:1467] Processing CreateTablet for tablet 6e70a5cf88d94b36913a6d1ccfcb979a (DEFAULT_TABLE table=default.Table_8e189703ccf14f70903828148ce964df [id=8dd3a08c5ec440389be4746b139a39aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.820593 5599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e70a5cf88d94b36913a6d1ccfcb979a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.825480 5600 tablet_service.cc:1514] Processing DeleteTablet for tablet daa59a18ae624bc8975bee2c460895a7 with delete_type TABLET_DATA_DELETED (replaced by tablet cb7f6487e28f4aaa94d448be71f8a3c2) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.826869 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 896389941179456d9d0ed3cd14726f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.827234 5609 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.828634 5615 tablet_service.cc:1467] Processing CreateTablet for tablet d2be108f26bb48d49216c6b7eef6b9b8 (DEFAULT_TABLE table=default.Table_cab0814763554410aa969d5b417710fb [id=eef293a0bbc94c2db94a64223ef71a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.835487 5345 tablet_service.cc:1467] Processing CreateTablet for tablet 9f53e5b5c4bd4a9385ad198b283c6e79 (DEFAULT_TABLE table=default.Table_22ad421714294dac9da7cb4984aee8f3 [id=0cdf67cb0e1f453e950a53dd444920de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.836475 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f53e5b5c4bd4a9385ad198b283c6e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.848451 5354 tablet_service.cc:1467] Processing CreateTablet for tablet d2be108f26bb48d49216c6b7eef6b9b8 (DEFAULT_TABLE table=default.Table_cab0814763554410aa969d5b417710fb [id=eef293a0bbc94c2db94a64223ef71a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.849853 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2be108f26bb48d49216c6b7eef6b9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.859560 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 45a26598a69146e1950e9227523fff13 (table default.Table_909de5e2ab404a69ba774963323e2102 [id=e85188ee407c472cbab5ab0ed13d27e6]) successfully deleted
I20260808 13:35:27.859822 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_93dd48e3ed114ba0a47115dac0dfcd7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:27.860245 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 973a7afdcbf944aab8f11b3fd3995865 because the tablet was not found. No further retry: Not found: Tablet not found: 973a7afdcbf944aab8f11b3fd3995865
W20260808 13:35:27.861369 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93dd48e3ed114ba0a47115dac0dfcd7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:27.861661 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6db6d1f1b6454336958e53566b8d48e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:27.862982 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 30) took 815 ms (client timeout 1000 ms). Trace:
W20260808 13:35:27.863420 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6db6d1f1b6454336958e53566b8d48e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:27.864485 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6529b65492c94717b809b0a0c5f23509"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:27.872985 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c7297f7910b4405ab0483761e1aed004"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:27.873698 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6529b65492c94717b809b0a0c5f23509 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:27.865784 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1665f65d290446e79ac5f6550ec733b7" } modify_external_catalogs: true
W20260808 13:35:27.874399 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7297f7910b4405ab0483761e1aed004 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:27.885864 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 6e70a5cf88d94b36913a6d1ccfcb979a (DEFAULT_TABLE table=default.Table_8e189703ccf14f70903828148ce964df [id=8dd3a08c5ec440389be4746b139a39aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.886900 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e70a5cf88d94b36913a6d1ccfcb979a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.894492 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1665f65d290446e79ac5f6550ec733b7" } modify_external_catalogs: true
I20260808 13:35:27.897825 4836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 3424ca753bcc4f448d314c56aba3323e
I20260808 13:35:27.899442 4836 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 285) took 2103 ms. Trace:
I20260808 13:35:27.899744 4836 rpcz_store.cc:276] 0808 13:35:25.796516 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.207263 (+1410747us) service_pool.cc:227] Handling call
0808 13:35:27.414489 (+207226us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:27.417024 (+ 2535us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:27.417157 (+ 133us) catalog_manager.cc:2761] Locking tablets
0808 13:35:27.694783 (+277626us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:27.897414 (+202631us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:27.897610 (+ 196us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:27.897736 (+ 126us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:27.899387 (+ 1651us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.705794 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:27.708087 (+ 2293us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:27.708114 (+ 27us) write_op.cc:435] Acquired schema lock
0808 13:35:27.708129 (+ 15us) tablet.cc:660] Decoding operations
0808 13:35:27.708791 (+ 662us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:27.708843 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:27.708912 (+ 69us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:27.709367 (+ 455us) tablet.cc:699] Row locks acquired
0808 13:35:27.709409 (+ 42us) write_op.cc:260] PREPARE: finished
0808 13:35:27.709608 (+ 199us) write_op.cc:270] Start()
0808 13:35:27.709755 (+ 147us) write_op.cc:276] Timestamp: P: 1786196127709547 usec, L: 0
0808 13:35:27.709771 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.713544 (+ 3773us) log.cc:842] Serialized 756 byte log entry
0808 13:35:27.893558 (+180014us) op_driver.cc:444] REPLICATION: finished
0808 13:35:27.894246 (+ 688us) write_op.cc:301] APPLY: starting
0808 13:35:27.894304 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:27.894411 (+ 107us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:27.894493 (+ 82us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:27.894797 (+ 304us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:27.894930 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:27.894971 (+ 41us) write_op.cc:312] APPLY: finished
0808 13:35:27.895139 (+ 168us) log.cc:842] Serialized 29 byte log entry
0808 13:35:27.895303 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.895391 (+ 88us) write_op.cc:454] Released schema lock
0808 13:35:27.895425 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:27.895585 (+ 160us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.895606 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":526,"apply.run_cpu_time_us":1552,"apply.run_wall_time_us":1553,"mutex_wait_us":6250,"num_ops":2,"prepare.queue_time_us":8725,"prepare.run_cpu_time_us":3562,"prepare.run_wall_time_us":159665,"raft.queue_time_us":8689,"raft.run_cpu_time_us":721,"raft.run_wall_time_us":721,"replication_time_us":183684}]]}
W20260808 13:35:27.870949 4832 rpcz_store.cc:269] 0808 13:35:27.047470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.095405 (+ 47935us) service_pool.cc:227] Handling call
0808 13:35:27.862921 (+767516us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.608098 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:27.608590 (+ 492us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:27.608619 (+ 29us) write_op.cc:435] Acquired schema lock
0808 13:35:27.608638 (+ 19us) tablet.cc:660] Decoding operations
0808 13:35:27.609073 (+ 435us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:27.609131 (+ 58us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:27.609170 (+ 39us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:27.609344 (+ 174us) tablet.cc:699] Row locks acquired
0808 13:35:27.609359 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:27.609467 (+ 108us) write_op.cc:270] Start()
0808 13:35:27.609554 (+ 87us) write_op.cc:276] Timestamp: P: 1786196127609379 usec, L: 0
0808 13:35:27.609573 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.610087 (+ 514us) log.cc:842] Serialized 356 byte log entry
0808 13:35:27.621952 (+ 11865us) op_driver.cc:444] REPLICATION: finished
0808 13:35:27.622334 (+ 382us) write_op.cc:301] APPLY: starting
0808 13:35:27.622399 (+ 65us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:27.622519 (+ 120us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:27.622650 (+ 131us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:27.622839 (+ 189us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:27.623021 (+ 182us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:27.623081 (+ 60us) write_op.cc:312] APPLY: finished
0808 13:35:27.623289 (+ 208us) log.cc:842] Serialized 23 byte log entry
0808 13:35:27.623422 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.623489 (+ 67us) write_op.cc:454] Released schema lock
0808 13:35:27.623527 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:27.623743 (+ 216us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.623775 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":150711,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":207,"apply.run_cpu_time_us":1622,"apply.run_wall_time_us":1622,"num_ops":1,"prepare.queue_time_us":63965,"prepare.run_cpu_time_us":2337,"prepare.run_wall_time_us":23826,"raft.queue_time_us":223,"raft.run_cpu_time_us":769,"raft.run_wall_time_us":770,"replication_time_us":12309}]]}
I20260808 13:35:27.930845 4832 master_service.cc:946] ReplaceTablet: received request to replace tablet 3a27517560d64a3b8da2cf55980e18a8 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:27.405391 5644 raft_consensus.cc:2454] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:27.826083 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 9f53e5b5c4bd4a9385ad198b283c6e79 (DEFAULT_TABLE table=default.Table_22ad421714294dac9da7cb4984aee8f3 [id=0cdf67cb0e1f453e950a53dd444920de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:27.905020 5599 tablet_service.cc:1514] Processing DeleteTablet for tablet 3424ca753bcc4f448d314c56aba3323e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:27 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:27.935526 5285 consensus_peers.cc:487] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 45a26598a69146e1950e9227523fff13. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:27.930991 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 with delete_type TABLET_DATA_DELETED (replaced by tablet d8b07b7d31ef41e484389c0f007598f0) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:27.931262 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2be108f26bb48d49216c6b7eef6b9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.931972 5616 tablet_service.cc:1514] Processing DeleteTablet for tablet b314734698ed423ba9cf4134db5846ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:27 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:27.939530 4823 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:37135, user_credentials={real_user=slave}} blocked reactor thread for 74781.7us
I20260808 13:35:27.943197 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_f28775e46ccb40e68c299486928fa702" } new_table_name: "default.Table_7fd510a91deb449091ff7da8c644fae1" modify_external_catalogs: true
I20260808 13:35:27.962809 4836 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_9769e23d59f2498e925cd703900ecbb3" } new_table_name: "default.Table_660efdc735cf4c31bf966be22b38dc28" modify_external_catalogs: true
I20260808 13:35:27.971786 4832 master_service.cc:946] ReplaceTablet: received request to replace tablet 5d388589d386408bb10ca030b87af787 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:27.983099 7032 raft_consensus.cc:482] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:27.984021 7032 raft_consensus.cc:504] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.987170 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet 3424ca753bcc4f448d314c56aba3323e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:27 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:27.988044 6046 consensus_queue.cc:260] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:27.990921 6046 ts_tablet_manager.cc:1428] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.219s user 0.006s sys 0.001s
I20260808 13:35:27.998245 6026 tablet_bootstrap.cc:492] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:28.008097 7208 raft_consensus.cc:482] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:28.009624 5636 consensus_meta.cc:287] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.663s user 0.000s sys 0.008s
I20260808 13:35:28.015691 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 9260370cf8cb46a7b699581e4e8d46db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:28.022002 7032 leader_election.cc:288] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:28.023097 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f53e5b5c4bd4a9385ad198b283c6e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:27.786870 7199 log.cc:1196] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/2cd2d0c9ab2c4704a74d06559bb913d6
I20260808 13:35:28.026940 5636 raft_consensus.cc:2454] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:28.028139 7014 raft_consensus.cc:2229] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:28.030923 7208 raft_consensus.cc:504] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.033062 7208 leader_election.cc:288] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:28.033571 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a26598a69146e1950e9227523fff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.034166 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e with delete_type TABLET_DATA_DELETED (replaced by tablet ba39706cff65404485959865fff1772a) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.034890 7014 raft_consensus.cc:2258] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:28.037319 6026 ts_tablet_manager.cc:1397] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.754s user 0.019s sys 0.019s
I20260808 13:35:28.036772 5607 tablet_service.cc:1467] Processing CreateTablet for tablet 342027968b884f1cbc78b1371318dfc9 (DEFAULT_TABLE table=default.Table_6ea0a150e9a245babc374b7222ad1ca7 [id=c69587f42e4b437aadf52a706e37e070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.047730 7049 consensus_queue.cc:1035] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:28.049108 7049 consensus_queue.cc:1035] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:28.051380 5603 consensus_meta.cc:287] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.750s user 0.010s sys 0.000s
W20260808 13:35:28.052989 5606 consensus_meta.cc:287] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.763s user 0.008s sys 0.002s
W20260808 13:35:28.053087 5285 consensus_peers.cc:487] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9260370cf8cb46a7b699581e4e8d46db. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:28.046051 6026 raft_consensus.cc:348] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.058924 6026 raft_consensus.cc:374] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:28.064332 6026 raft_consensus.cc:729] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:28.075183 7014 ts_tablet_manager.cc:1905] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:28.034742 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.084375 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet e992a0294c404cfb9541b57782a05fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:27.940373 4823 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2d8e4af746774747a70d425068e8dd63 (table default.Table_256bb136799848f4891c3923692e5feb [id=95f59bd3447c44c291270c448f53d78b], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2d8e4af746774747a70d425068e8dd63 consensus state has no leader
I20260808 13:35:28.089656 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 with delete_type TABLET_DATA_DELETED (replaced by tablet d8b07b7d31ef41e484389c0f007598f0) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.090227 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 896389941179456d9d0ed3cd14726f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.091042 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.091526 6913 raft_consensus.cc:482] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:28.092376 6913 raft_consensus.cc:504] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.096246 6913 leader_election.cc:288] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:28.100809 5772 leader_election.cc:302] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:28.070991 6026 consensus_queue.cc:260] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.103097 5772 leader_election.cc:302] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:28.104955 6901 raft_consensus.cc:482] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:28.041668 7199 ts_tablet_manager.cc:1939] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:28.105379 6901 raft_consensus.cc:504] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.107570 6901 leader_election.cc:288] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:28.109229 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3682c9c656c4556bc13ae3aeddd7c5e" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:28.117479 5881 raft_consensus.cc:2379] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:28.117857 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 3424ca753bcc4f448d314c56aba3323e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:27 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.118657 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 9260370cf8cb46a7b699581e4e8d46db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.127868 7014 ts_tablet_manager.cc:1918] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:28.128708 7014 log.cc:1196] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/d3682c9c656c4556bc13ae3aeddd7c5e
I20260808 13:35:28.129320 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 2cd2d0c9ab2c4704a74d06559bb913d6 (table default.Table_34821e686c9b4e988ffdbb94388b2280 [id=ccd192d15e9d42b5ab0739dc606301e1]) successfully deleted
I20260808 13:35:28.131636 7241 raft_consensus.cc:2781] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
W20260808 13:35:28.134235 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 121171us
I20260808 13:35:28.132174 6901 raft_consensus.cc:482] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:28.135610 6901 raft_consensus.cc:504] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.137565 6901 leader_election.cc:288] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:28.162751 7014 ts_tablet_manager.cc:1939] T d3682c9c656c4556bc13ae3aeddd7c5e P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:28.137102 7197 raft_consensus.cc:2781] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:28.168335 7197 raft_consensus.cc:686] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:28.169226 7197 consensus_queue.cc:237] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.139261 7199 tablet_replica.cc:330] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:28.172202 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_321b4eb3dd024925af1cb354d3fe1567" } modify_external_catalogs: true
I20260808 13:35:28.172739 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_321b4eb3dd024925af1cb354d3fe1567" } modify_external_catalogs: true
I20260808 13:35:28.175890 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 4625b632d67244fc86e48d739dde707e
I20260808 13:35:28.178094 7199 raft_consensus.cc:2229] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:28.178910 7199 raft_consensus.cc:2258] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:28.180948 7199 ts_tablet_manager.cc:1905] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:28.186197 5845 tablet_service.cc:1514] Processing DeleteTablet for tablet a541ec35b1a44109866fabc953f14cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.186991 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 4625b632d67244fc86e48d739dde707e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:27 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.167243 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3682c9c656c4556bc13ae3aeddd7c5e" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:28.172094 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342027968b884f1cbc78b1371318dfc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.180150 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 4625b632d67244fc86e48d739dde707e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:27 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:28.197615 5285 leader_election.cc:341] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: Tablet not RUNNING: SHUTDOWN
I20260808 13:35:28.198064 5285 leader_election.cc:302] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:28.199378 7219 raft_consensus.cc:3037] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:28.211264 7241 raft_consensus.cc:686] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:28.214062 7219 raft_consensus.cc:2726] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:28.214274 6033 tablet_bootstrap.cc:654] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:28.217666 4836 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 42218e4330d147f389717108bfe0acea consensus state has no leader
I20260808 13:35:28.219218 6046 tablet_bootstrap.cc:492] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:28.224684 5603 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50600 (request call id 242) took 1005 ms. Trace:
I20260808 13:35:28.225088 5603 rpcz_store.cc:276] 0808 13:35:27.219682 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.219847 (+ 165us) service_pool.cc:227] Handling call
0808 13:35:27.254857 (+ 35010us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:27.254936 (+ 79us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:27.301201 (+ 46265us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:28.052118 (+750917us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:28.224603 (+172485us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:28.228188 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet 4625b632d67244fc86e48d739dde707e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:27 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:28.228281 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_280edf174ae84743aac082cfd2788315"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:28.229810 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_280edf174ae84743aac082cfd2788315 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:28.233009 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_577d84019e77413d9ff7373a8315d384"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:28.234449 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_577d84019e77413d9ff7373a8315d384 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:28.241411 5283 consensus_peers.cc:487] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 45a26598a69146e1950e9227523fff13. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:28.246912 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127e8d0323954ef788018bcb91582951" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:28.261205 6046 tablet_bootstrap.cc:654] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:28.264768 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 236) took 1139 ms. Trace:
I20260808 13:35:28.265120 7014 rpcz_store.cc:276] 0808 13:35:27.124915 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.126371 (+ 1456us) service_pool.cc:227] Handling call
0808 13:35:27.394062 (+267691us) ts_tablet_manager.cc:1145] Deleting tablet d3682c9c656c4556bc13ae3aeddd7c5e
0808 13:35:27.394100 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.034561 (+640461us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54000c1a54. stack: 00007f5a54110033 00007f5a5410ff30 00007f5a53c91534 00000000004db2b4 00000000004db249 00007f5a5b01c471 00007f5a5b09f352 00007f5a5b4b526c 00007f5a5c0c2cfa 00007f5a5c0f2e10 00007f5a5c0d9e0d 00007f5a5c0d9db9 00007f5a5c0d9d49 00007f5a5c0d9d11 00007f5a5c0d8f3d 0000000000580784
0808 13:35:28.116037 (+ 81476us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:28.127460 (+ 11423us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:28.264700 (+137240us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1223913344,"tablet-delete.queue_time_us":266897}
I20260808 13:35:28.265789 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f2e0b98af431d99894e8778257bc4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:28.266775 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet d3682c9c656c4556bc13ae3aeddd7c5e (table default.Table_fc7cb305e35541c781c3e3790cf8a0af [id=b885c5ab382c45809d6f896ebf8771bd]) successfully deleted
I20260808 13:35:28.267393 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:28.270205 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf907a9f19f4d64b27be946011a8ddb" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:28.268329 5603 tablet_service.cc:1467] Processing CreateTablet for tablet e1d2aae6405349fc887ec8527464e46a (DEFAULT_TABLE table=default.Table_33484efb632a4860bc2485f6713c84d0 [id=26126642d65b4c8bbacd0e4741ed7eee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.271514 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d2aae6405349fc887ec8527464e46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.271849 5608 tablet_service.cc:1467] Processing CreateTablet for tablet cb7f6487e28f4aaa94d448be71f8a3c2 (DEFAULT_TABLE table=default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.273314 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb7f6487e28f4aaa94d448be71f8a3c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.283892 5836 tablet_service.cc:1467] Processing CreateTablet for tablet e1d2aae6405349fc887ec8527464e46a (DEFAULT_TABLE table=default.Table_33484efb632a4860bc2485f6713c84d0 [id=26126642d65b4c8bbacd0e4741ed7eee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.284971 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d2aae6405349fc887ec8527464e46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.287326 4832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 5d388589d386408bb10ca030b87af787
I20260808 13:35:28.287400 5615 tablet_service.cc:1467] Processing CreateTablet for tablet aa493b8652e149f988d41bee811f6a2f (DEFAULT_TABLE table=default.Table_52f5a796f61f4be79a196bc23065491b [id=ee175d53df794e6aaf29faff1fc2bb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.287619 5336 tablet_service.cc:1467] Processing CreateTablet for tablet e1d2aae6405349fc887ec8527464e46a (DEFAULT_TABLE table=default.Table_33484efb632a4860bc2485f6713c84d0 [id=26126642d65b4c8bbacd0e4741ed7eee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.288470 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa493b8652e149f988d41bee811f6a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.288873 4832 catalog_manager.cc:6565] ReplaceTablet: tablet 5d388589d386408bb10ca030b87af787 deleted and replaced by tablet 3ab26fa1872e4acf874f6577bded5e43
I20260808 13:35:28.289011 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d2aae6405349fc887ec8527464e46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.293784 5345 tablet_service.cc:1467] Processing CreateTablet for tablet cb7f6487e28f4aaa94d448be71f8a3c2 (DEFAULT_TABLE table=default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.294754 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb7f6487e28f4aaa94d448be71f8a3c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.300064 5334 tablet_service.cc:1467] Processing CreateTablet for tablet aa493b8652e149f988d41bee811f6a2f (DEFAULT_TABLE table=default.Table_52f5a796f61f4be79a196bc23065491b [id=ee175d53df794e6aaf29faff1fc2bb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.300463 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 61dfd47db0a74ed8b7228b559dd68459 (DEFAULT_TABLE table=default.Table_a7a4e8c009fd4046ad8a17c30d9c4de5 [id=948a64c5af6f44b7bbce702a2d965a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.301151 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa493b8652e149f988d41bee811f6a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.301849 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61dfd47db0a74ed8b7228b559dd68459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.310946 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d388589d386408bb10ca030b87af787 with delete_type TABLET_DATA_DELETED (replaced by tablet 3ab26fa1872e4acf874f6577bded5e43) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:28.317113 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 604d2113451642329b877f8beff71618 (DEFAULT_TABLE table=default.Table_6c19755f3f9149e7bdac49809abbe2a3 [id=bd6ed2fb84f44308a249c7b2cfa8d6a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.361057 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a8ff8993913d4a05a3d8ab3bb339b7ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:28.363755 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8ff8993913d4a05a3d8ab3bb339b7ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:28.372433 5551 leader_election.cc:341] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:28.372927 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 96693.4us
W20260808 13:35:28.373847 5551 leader_election.cc:341] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:28.373909 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d388589d386408bb10ca030b87af787 with delete_type TABLET_DATA_DELETED (replaced by tablet 3ab26fa1872e4acf874f6577bded5e43) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:28.376965 5845 tablet_service.cc:1467] Processing CreateTablet for tablet cb7f6487e28f4aaa94d448be71f8a3c2 (DEFAULT_TABLE table=default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.377974 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb7f6487e28f4aaa94d448be71f8a3c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.378530 7261 raft_consensus.cc:482] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:28.378957 7261 raft_consensus.cc:504] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.380214 5834 tablet_service.cc:1467] Processing CreateTablet for tablet aa493b8652e149f988d41bee811f6a2f (DEFAULT_TABLE table=default.Table_52f5a796f61f4be79a196bc23065491b [id=ee175d53df794e6aaf29faff1fc2bb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.380523 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 604d2113451642329b877f8beff71618 (DEFAULT_TABLE table=default.Table_6c19755f3f9149e7bdac49809abbe2a3 [id=bd6ed2fb84f44308a249c7b2cfa8d6a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.381234 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa493b8652e149f988d41bee811f6a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.381532 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 604d2113451642329b877f8beff71618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.383219 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 61dfd47db0a74ed8b7228b559dd68459 (DEFAULT_TABLE table=default.Table_a7a4e8c009fd4046ad8a17c30d9c4de5 [id=948a64c5af6f44b7bbce702a2d965a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.374320 5606 tablet_service.cc:1467] Processing CreateTablet for tablet 61dfd47db0a74ed8b7228b559dd68459 (DEFAULT_TABLE table=default.Table_a7a4e8c009fd4046ad8a17c30d9c4de5 [id=948a64c5af6f44b7bbce702a2d965a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.383782 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_22ad421714294dac9da7cb4984aee8f3" } modify_external_catalogs: true
I20260808 13:35:28.384132 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61dfd47db0a74ed8b7228b559dd68459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.384251 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61dfd47db0a74ed8b7228b559dd68459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.383773 7241 consensus_queue.cc:237] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.384294 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_22ad421714294dac9da7cb4984aee8f3" } modify_external_catalogs: true
I20260808 13:35:28.387261 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 604d2113451642329b877f8beff71618 (DEFAULT_TABLE table=default.Table_6c19755f3f9149e7bdac49809abbe2a3 [id=bd6ed2fb84f44308a249c7b2cfa8d6a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:28.388186 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 604d2113451642329b877f8beff71618. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:28.374347 5551 leader_election.cc:341] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:28.391680 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_41a67a20ea754c6fb307fb434f9711ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:28.393138 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41a67a20ea754c6fb307fb434f9711ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:28.395201 5551 leader_election.cc:341] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:28.397989 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c7d6c5128772436194a73fb2f48105b4" } new_table_name: "default.Table_b339b57a697d413ab7d9ec93220be980" modify_external_catalogs: true
I20260808 13:35:28.406885 7254 consensus_queue.cc:1035] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260808 13:35:28.408190 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d388589d386408bb10ca030b87af787 with delete_type TABLET_DATA_DELETED (replaced by tablet 3ab26fa1872e4acf874f6577bded5e43) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.408006 7014 tablet_replica.cc:330] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:28.415810 6026 ts_tablet_manager.cc:1428] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.378s user 0.007s sys 0.013s
I20260808 13:35:28.416253 4841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet a8e18e51c2e24d3caa3d29e6645e46ae
I20260808 13:35:28.416983 6026 tablet_bootstrap.cc:492] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:28.421934 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e18e51c2e24d3caa3d29e6645e46ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.422842 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e18e51c2e24d3caa3d29e6645e46ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:28.423326 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_04a268476ef54bb68aadad0d39d6e838" } new_table_name: "default.Table_f676fb75c0ba45a98625440ed5dd7313" modify_external_catalogs: true
I20260808 13:35:28.430732 4840 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_7a1963a007954f8ca1d330f9d1936dd3 [id=d5b4528cac46486e857db6176b4b3a2b] alter complete (version 3)
W20260808 13:35:28.444442 5283 consensus_peers.cc:487] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2cd2d0c9ab2c4704a74d06559bb913d6. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:28.451327 4833 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 47) took 1007 ms (client timeout 1000 ms). Trace:
I20260808 13:35:28.452592 7246 tablet.cc:1722] T 1947f03e107447bb8e23fe5deaf1ea60 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:28.467912 7014 raft_consensus.cc:2229] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:28.469703 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 604d2113451642329b877f8beff71618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:28.472471 5476 heartbeater.cc:347] Connected to a master server at 127.4.170.62:34069
I20260808 13:35:28.475883 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e18e51c2e24d3caa3d29e6645e46ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:28.480125 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 with delete_type TABLET_DATA_DELETED (replaced by tablet d8b07b7d31ef41e484389c0f007598f0) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:28.482699 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 2173e97590be43f9a24f55689d796269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:28.488669 7013 ts_tablet_manager.cc:1918] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:28.489300 7013 log.cc:1196] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/973a7afdcbf944aab8f11b3fd3995865
I20260808 13:35:28.511104 7014 raft_consensus.cc:2258] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:28.511900 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f2e0b98af431d99894e8778257bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:28.490367 7013 ts_tablet_manager.cc:1939] T 973a7afdcbf944aab8f11b3fd3995865 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:28.529117 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 223) took 3859 ms. Trace:
I20260808 13:35:28.539383 7013 rpcz_store.cc:276] 0808 13:35:24.669902 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.686579 (+ 16677us) service_pool.cc:227] Handling call
0808 13:35:27.605708 (+2919129us) ts_tablet_manager.cc:1145] Deleting tablet 973a7afdcbf944aab8f11b3fd3995865
0808 13:35:27.605752 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.146919 (+541167us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:28.488316 (+341397us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:28.529050 (+ 40734us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4365,"tablet-delete.queue_time_us":2917280}
W20260808 13:35:28.483906 4833 rpcz_store.cc:269] 0808 13:35:27.443473 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.484752 (+ 41279us) service_pool.cc:227] Handling call
0808 13:35:27.500560 (+ 15808us) spinlock_profiling.cc:243] Waited 9.86 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:28.451246 (+950686us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.865550 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:27.865988 (+ 438us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:27.866012 (+ 24us) write_op.cc:435] Acquired schema lock
0808 13:35:27.866023 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:27.866336 (+ 313us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:27.866381 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:27.866411 (+ 30us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:27.866574 (+ 163us) tablet.cc:699] Row locks acquired
0808 13:35:27.866592 (+ 18us) write_op.cc:260] PREPARE: finished
0808 13:35:27.866765 (+ 173us) write_op.cc:270] Start()
0808 13:35:27.866850 (+ 85us) write_op.cc:276] Timestamp: P: 1786196127866685 usec, L: 0
0808 13:35:27.866865 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.867392 (+ 527us) log.cc:842] Serialized 527 byte log entry
0808 13:35:27.893794 (+ 26402us) op_driver.cc:444] REPLICATION: finished
0808 13:35:27.895921 (+ 2127us) write_op.cc:301] APPLY: starting
0808 13:35:27.895970 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:27.896049 (+ 79us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:27.896127 (+ 78us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:27.896298 (+ 171us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:27.896414 (+ 116us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:27.896454 (+ 40us) write_op.cc:312] APPLY: finished
0808 13:35:27.896604 (+ 150us) log.cc:842] Serialized 23 byte log entry
0808 13:35:27.896726 (+ 122us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.896790 (+ 64us) write_op.cc:454] Released schema lock
0808 13:35:27.896819 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:27.896979 (+ 160us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:27.897003 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8593,"reported_tablets":28,"spinlock_wait_cycles":22679424,"child_traces":[["op",{"apply.queue_time_us":1994,"apply.run_cpu_time_us":1220,"apply.run_wall_time_us":1220,"num_ops":1,"prepare.queue_time_us":106538,"prepare.run_cpu_time_us":2096,"prepare.run_wall_time_us":2106,"raft.queue_time_us":987,"raft.run_cpu_time_us":13,"raft.run_wall_time_us":13,"replication_time_us":26852,"thread_start_us":46582,"threads_started":1}]]}
I20260808 13:35:28.544365 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 127e8d0323954ef788018bcb91582951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:28.567426 7262 log.cc:924] Time spent T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Append to log took a long time: real 0.114s user 0.001s sys 0.000s
I20260808 13:35:28.587100 7269 tablet.cc:1722] T 1947f03e107447bb8e23fe5deaf1ea60 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:28.613087 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:28.614066 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 224) took 3851 ms. Trace:
I20260808 13:35:28.613837 7192 tablet.cc:1722] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:28.614423 7013 rpcz_store.cc:276] 0808 13:35:24.762758 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:24.762878 (+ 120us) service_pool.cc:227] Handling call
0808 13:35:28.543851 (+3780973us) ts_tablet_manager.cc:1145] Deleting tablet 896389941179456d9d0ed3cd14726f11
0808 13:35:28.567120 (+ 23269us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.614010 (+ 46890us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3748747}
W20260808 13:35:28.614832 5771 leader_election.cc:341] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:28.615676 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 896389941179456d9d0ed3cd14726f11 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 896389941179456d9d0ed3cd14726f11 already in progress: creating tablet
I20260808 13:35:28.618824 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 228) took 3058 ms. Trace:
I20260808 13:35:28.622690 7013 rpcz_store.cc:276] 0808 13:35:25.560099 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:25.562488 (+ 2389us) service_pool.cc:227] Handling call
0808 13:35:28.615381 (+3052893us) ts_tablet_manager.cc:1145] Deleting tablet 2cd6945ecccc4684a3fa5b2a1f07057f
0808 13:35:28.618221 (+ 2840us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.618744 (+ 523us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3032791}
W20260808 13:35:28.632221 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 2cd6945ecccc4684a3fa5b2a1f07057f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2cd6945ecccc4684a3fa5b2a1f07057f already in progress: creating tablet
I20260808 13:35:28.646263 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_ff9135dbb8a84d158dc9f2f209d8d741"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:28.647898 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff9135dbb8a84d158dc9f2f209d8d741 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:28.651337 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_323ff643b74d4f8e8a82e3fe8d1fc10c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:28.652781 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_323ff643b74d4f8e8a82e3fe8d1fc10c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:28.681767 4835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a730d953de56401597f5e28a6edc707a" } modify_external_catalogs: true
I20260808 13:35:28.682297 4835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a730d953de56401597f5e28a6edc707a" } modify_external_catalogs: true
I20260808 13:35:28.691056 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 229) took 3096 ms. Trace:
I20260808 13:35:28.691416 7013 rpcz_store.cc:276] 0808 13:35:25.595000 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:25.595118 (+ 118us) service_pool.cc:227] Handling call
0808 13:35:28.633415 (+3038297us) ts_tablet_manager.cc:1145] Deleting tablet 362dbd6990e740b39690537af98aa2f8
0808 13:35:28.690710 (+ 57295us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.690995 (+ 285us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3029784}
I20260808 13:35:28.692785 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 246) took 1236 ms. Trace:
I20260808 13:35:28.692775 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:28.693137 7013 rpcz_store.cc:276] 0808 13:35:27.456848 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.458347 (+ 1499us) service_pool.cc:227] Handling call
0808 13:35:28.692422 (+1234075us) ts_tablet_manager.cc:1145] Deleting tablet a0b29786cb404bc2b01473c3b8390a50
0808 13:35:28.692469 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.692727 (+ 258us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1233563}
I20260808 13:35:28.694000 6046 tablet_bootstrap.cc:492] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
W20260808 13:35:28.695780 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 362dbd6990e740b39690537af98aa2f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 362dbd6990e740b39690537af98aa2f8 already in progress: creating tablet
W20260808 13:35:28.696544 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet a0b29786cb404bc2b01473c3b8390a50 because the tablet was not found. No further retry: Not found: Tablet not found: a0b29786cb404bc2b01473c3b8390a50
I20260808 13:35:28.694372 6046 ts_tablet_manager.cc:1397] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.475s user 0.009s sys 0.002s
I20260808 13:35:28.694336 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 249) took 1235 ms. Trace:
I20260808 13:35:28.707592 6046 raft_consensus.cc:348] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.723418 6046 raft_consensus.cc:374] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:28.723912 6046 raft_consensus.cc:729] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:28.724974 6046 consensus_queue.cc:260] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.715664 7013 rpcz_store.cc:276] 0808 13:35:27.458629 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.458818 (+ 189us) service_pool.cc:227] Handling call
0808 13:35:28.693987 (+1235169us) ts_tablet_manager.cc:1145] Deleting tablet daa59a18ae624bc8975bee2c460895a7
0808 13:35:28.694026 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.694292 (+ 266us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1234670}
I20260808 13:35:28.749650 7192 tablet.cc:1722] T f26e054d1992452396d3f7334ba3602c P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:28.750340 7013 tablet_replica.cc:330] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:28.751299 7013 raft_consensus.cc:2229] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:28.751775 7013 raft_consensus.cc:2258] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:28.752758 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet daa59a18ae624bc8975bee2c460895a7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet daa59a18ae624bc8975bee2c460895a7 already in progress: creating tablet
I20260808 13:35:28.751776 6046 ts_tablet_manager.cc:1428] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.054s user 0.005s sys 0.001s
I20260808 13:35:28.765487 6046 tablet_bootstrap.cc:492] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:28.771123 6046 tablet_bootstrap.cc:654] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:28.774463 7013 ts_tablet_manager.cc:1905] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:28.803149 4839 master_service.cc:946] ReplaceTablet: received request to replace tablet 973a7afdcbf944aab8f11b3fd3995865 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:28.805902 6033 tablet_bootstrap.cc:492] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:28.806336 6033 ts_tablet_manager.cc:1397] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 1.356s user 0.008s sys 0.008s
I20260808 13:35:28.822548 6026 tablet_bootstrap.cc:654] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:28.823760 7014 ts_tablet_manager.cc:1905] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:28.824268 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 9f53e5b5c4bd4a9385ad198b283c6e79
W20260808 13:35:28.826229 7277 log.cc:924] Time spent T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Append to log took a long time: real 0.182s user 0.000s sys 0.000s
W20260808 13:35:28.827210 4837 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a73e98a8b20144508a5a8d451f4bb302 consensus state has no leader
I20260808 13:35:28.834362 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f53e5b5c4bd4a9385ad198b283c6e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:28.838932 6046 tablet_bootstrap.cc:492] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:28.839377 6046 ts_tablet_manager.cc:1397] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.074s user 0.012s sys 0.002s
W20260808 13:35:28.839442 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eaa049ef916b44458be43ec4e2ccaf1c consensus state has no leader
I20260808 13:35:28.835450 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:28.847211 5636 raft_consensus.cc:2454] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:28.864127 7013 ts_tablet_manager.cc:1918] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:28.865295 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f53e5b5c4bd4a9385ad198b283c6e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:28.873766 7013 log.cc:1196] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:28.862272 6046 raft_consensus.cc:348] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.874838 6046 raft_consensus.cc:374] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:28.875008 7013 ts_tablet_manager.cc:1939] T 81fb3e099f7f4544b3c100070f0d14ef P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:28.875154 6046 raft_consensus.cc:729] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:28.876267 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 248) took 1418 ms. Trace:
I20260808 13:35:28.876858 7013 rpcz_store.cc:276] 0808 13:35:27.458313 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.458777 (+ 464us) service_pool.cc:227] Handling call
0808 13:35:28.747675 (+1288898us) ts_tablet_manager.cc:1145] Deleting tablet 81fb3e099f7f4544b3c100070f0d14ef
0808 13:35:28.750176 (+ 2501us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.839440 (+ 89264us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:28.863574 (+ 24134us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:28.876179 (+ 12605us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1287360}
I20260808 13:35:28.878762 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 81fb3e099f7f4544b3c100070f0d14ef (table default.Table_76c233d2deef48ec8ec2fba9d29f6ecb [id=bdb7331763214cbe82fe544204f8b2a3]) successfully deleted
I20260808 13:35:28.875969 6046 consensus_queue.cc:260] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.886094 6046 ts_tablet_manager.cc:1428] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.046s user 0.005s sys 0.002s
I20260808 13:35:28.887094 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_b5830adfb9064ab4b32c29c4c1a03066"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:28.888815 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5830adfb9064ab4b32c29c4c1a03066 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:28.886849 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 245) took 1430 ms. Trace:
I20260808 13:35:28.897179 7013 rpcz_store.cc:276] 0808 13:35:27.456578 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.460089 (+ 3511us) service_pool.cc:227] Handling call
0808 13:35:28.886568 (+1426479us) ts_tablet_manager.cc:1145] Deleting tablet b809b185ee9e4c658a050c41c9abc4ad
0808 13:35:28.886639 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.886804 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1426025}
I20260808 13:35:28.901549 6046 tablet_bootstrap.cc:492] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:28.902894 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet b809b185ee9e4c658a050c41c9abc4ad because the tablet was not found. No further retry: Not found: Tablet not found: b809b185ee9e4c658a050c41c9abc4ad
I20260808 13:35:28.915474 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f53e5b5c4bd4a9385ad198b283c6e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:28.919955 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 250) took 1451 ms. Trace:
I20260808 13:35:28.920395 7013 rpcz_store.cc:276] 0808 13:35:27.468475 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.468654 (+ 179us) service_pool.cc:227] Handling call
0808 13:35:28.901078 (+1432424us) ts_tablet_manager.cc:1145] Deleting tablet b314734698ed423ba9cf4134db5846ec
0808 13:35:28.919352 (+ 18274us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.919878 (+ 526us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1431881}
W20260808 13:35:28.920799 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 108794us
I20260808 13:35:28.921599 6901 raft_consensus.cc:482] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:28.921927 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet b314734698ed423ba9cf4134db5846ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b314734698ed423ba9cf4134db5846ec already in progress: creating tablet
I20260808 13:35:28.922974 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 242) took 1467 ms. Trace:
I20260808 13:35:28.923312 7013 rpcz_store.cc:276] 0808 13:35:27.455709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.468669 (+ 12960us) service_pool.cc:227] Handling call
0808 13:35:28.922701 (+1454032us) ts_tablet_manager.cc:1145] Deleting tablet 543debff6da54d26bf62260bf8721ff7
0808 13:35:28.922741 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.922911 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1453459}
W20260808 13:35:28.924607 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 543debff6da54d26bf62260bf8721ff7 because the tablet was not found. No further retry: Not found: Tablet not found: 543debff6da54d26bf62260bf8721ff7
I20260808 13:35:28.925554 7013 tablet_replica.cc:330] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:28.928150 4833 master_service.cc:946] ReplaceTablet: received request to replace tablet 31fc9fdb5ef543da833f9af6ca5807a9 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:28.928864 6026 tablet_bootstrap.cc:492] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:28.922166 6901 raft_consensus.cc:504] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.928980 7013 raft_consensus.cc:2229] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:28.929298 6026 ts_tablet_manager.cc:1397] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.512s user 0.015s sys 0.001s
I20260808 13:35:28.930058 7013 raft_consensus.cc:2258] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:28.932276 7013 ts_tablet_manager.cc:1905] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:28.933135 6901 leader_election.cc:288] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:28.933362 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fc9fdb5ef543da833f9af6ca5807a9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:28.937117 7192 tablet.cc:1722] T c8f1618a1d044898bffbc82107e90840 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:28.940069 6046 tablet_bootstrap.cc:654] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:28.940371 5551 leader_election.cc:341] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:28.956338 7280 tablet.cc:1722] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:28.954783 7282 tablet.cc:1722] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260808 13:35:28.970227 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet eaa049ef916b44458be43ec4e2ccaf1c (table default.Table_f676fb75c0ba45a98625440ed5dd7313 [id=d342034fcd7b45debb1aca898c6b7f68], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet eaa049ef916b44458be43ec4e2ccaf1c consensus state has no leader
I20260808 13:35:28.972992 6046 tablet_bootstrap.cc:492] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:28.973417 6046 ts_tablet_manager.cc:1397] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.072s user 0.014s sys 0.001s
I20260808 13:35:28.976356 6046 raft_consensus.cc:348] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.804723 7261 leader_election.cc:288] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:28.977073 6046 raft_consensus.cc:374] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:28.977334 6046 raft_consensus.cc:729] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:28.978080 6046 consensus_queue.cc:260] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.983436 5772 leader_election.cc:302] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:28.960860 6026 raft_consensus.cc:348] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:28.984002 6026 raft_consensus.cc:374] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:28.984275 6026 raft_consensus.cc:729] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:28.987756 6046 ts_tablet_manager.cc:1428] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.014s user 0.002s sys 0.006s
I20260808 13:35:28.988291 7281 tablet.cc:1722] T f26e054d1992452396d3f7334ba3602c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:28.990999 6046 tablet_bootstrap.cc:492] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:28.992115 7279 log.cc:924] Time spent T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.326s user 0.001s sys 0.000s
I20260808 13:35:28.993897 7014 ts_tablet_manager.cc:1918] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:29.001464 7013 ts_tablet_manager.cc:1918] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:29.001864 7013 log.cc:1196] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/2cd2d0c9ab2c4704a74d06559bb913d6
I20260808 13:35:29.002599 7013 ts_tablet_manager.cc:1939] T 2cd2d0c9ab2c4704a74d06559bb913d6 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:29.015600 6046 tablet_bootstrap.cc:654] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:29.016606 7014 log.cc:1196] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/e992a0294c404cfb9541b57782a05fc9
I20260808 13:35:29.008131 4838 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:28.989964 6026 consensus_queue.cc:260] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.018458 4838 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_9d2867cdebf84440b7aa21a71faba9c4 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 2 got 0
I20260808 13:35:29.019110 4838 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet f26e054d1992452396d3f7334ba3602c (table default.Table_95ed126fb41342c1a6a1539f0a5ebbe3 [id=b301b3d4964b41009ed2903dcbefdfae]). Expected version 1 got 0
I20260808 13:35:29.025835 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 247) took 1568 ms. Trace:
I20260808 13:35:29.027112 7013 rpcz_store.cc:276] 0808 13:35:27.457098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.460621 (+ 3523us) service_pool.cc:227] Handling call
0808 13:35:28.925373 (+1464752us) ts_tablet_manager.cc:1145] Deleting tablet 2cd2d0c9ab2c4704a74d06559bb913d6
0808 13:35:28.925412 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.955644 (+ 30232us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:28.999535 (+ 43891us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.025768 (+ 26233us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1446317}
I20260808 13:35:29.028385 7013 tablet_replica.cc:330] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:29.032545 4832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_9a37007965804c1ea56673789a63e22b" } new_table_name: "default.Table_fbdbcba42ce04dd68c37a5f32605be9a" modify_external_catalogs: true
I20260808 13:35:29.033430 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 2cd2d0c9ab2c4704a74d06559bb913d6 (table default.Table_34821e686c9b4e988ffdbb94388b2280 [id=ccd192d15e9d42b5ab0739dc606301e1]) successfully deleted
I20260808 13:35:29.035004 6026 ts_tablet_manager.cc:1428] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.105s user 0.010s sys 0.003s
I20260808 13:35:29.032545 4838 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet c8f1618a1d044898bffbc82107e90840 (table default.Table_7fd510a91deb449091ff7da8c644fae1 [id=1b86c1f240214c88b29348f33a11503d]). Expected version 1 got 0
I20260808 13:35:29.039072 4838 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:29.043205 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_114737be7a5c48089f667a5f02f76b5e" } new_table_name: "default.Table_e0361960d323473f83c14c853ffcb172" modify_external_catalogs: true
I20260808 13:35:29.044315 7013 raft_consensus.cc:2229] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:29.045209 7013 raft_consensus.cc:2258] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:29.047053 6026 tablet_bootstrap.cc:492] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:29.049662 7013 ts_tablet_manager.cc:1905] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:29.050084 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5ede676dcd5843a3b896699541bf6d01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:29.051863 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ede676dcd5843a3b896699541bf6d01 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:29.037309 7014 ts_tablet_manager.cc:1939] T e992a0294c404cfb9541b57782a05fc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:29.064651 6046 tablet_bootstrap.cc:492] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:29.065063 6046 ts_tablet_manager.cc:1397] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.076s user 0.009s sys 0.009s
I20260808 13:35:29.068037 6046 raft_consensus.cc:348] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.068756 6046 raft_consensus.cc:374] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:29.069092 6046 raft_consensus.cc:729] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:29.075618 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_40a75c335146479fb7dc85888769c778" } modify_external_catalogs: true
I20260808 13:35:29.076227 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_40a75c335146479fb7dc85888769c778" } modify_external_catalogs: true
W20260808 13:35:29.080703 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 97388.8us
I20260808 13:35:29.084344 6046 consensus_queue.cc:260] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.091571 6913 raft_consensus.cc:482] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:29.092146 6913 raft_consensus.cc:504] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.094972 7013 ts_tablet_manager.cc:1918] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:29.101037 7013 log.cc:1196] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/45a26598a69146e1950e9227523fff13
I20260808 13:35:29.101975 7013 ts_tablet_manager.cc:1939] T 45a26598a69146e1950e9227523fff13 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:29.104029 6901 raft_consensus.cc:482] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:29.104861 6901 raft_consensus.cc:504] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.107077 6046 ts_tablet_manager.cc:1428] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.042s user 0.012s sys 0.002s
I20260808 13:35:29.108008 6046 tablet_bootstrap.cc:492] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:29.108548 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6eb3cce7943e47dc9749d45cfae01235"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:29.110327 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6eb3cce7943e47dc9749d45cfae01235 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:29.112926 6913 leader_election.cc:288] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:29.114508 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 240) took 1980 ms. Trace:
I20260808 13:35:29.114899 7014 rpcz_store.cc:276] 0808 13:35:27.128127 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.128231 (+ 104us) service_pool.cc:227] Handling call
0808 13:35:28.268692 (+1140461us) ts_tablet_manager.cc:1145] Deleting tablet e992a0294c404cfb9541b57782a05fc9
0808 13:35:28.407889 (+139197us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.898967 (+491078us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:28.993640 (+ 94673us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.108011 (+114371us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":52272,"tablet-delete.queue_time_us":1139798}
I20260808 13:35:29.116467 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 241) took 1677 ms. Trace:
I20260808 13:35:29.116808 7013 rpcz_store.cc:276] 0808 13:35:27.439244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.451621 (+ 12377us) service_pool.cc:227] Handling call
0808 13:35:29.028236 (+1576615us) ts_tablet_manager.cc:1145] Deleting tablet 45a26598a69146e1950e9227523fff13
0808 13:35:29.028279 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.057635 (+ 29356us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.094658 (+ 37023us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.116406 (+ 21748us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15161,"tablet-delete.queue_time_us":1541268}
I20260808 13:35:29.118191 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 45a26598a69146e1950e9227523fff13 (table default.Table_909de5e2ab404a69ba774963323e2102 [id=e85188ee407c472cbab5ab0ed13d27e6]) successfully deleted
I20260808 13:35:29.119076 7013 tablet_replica.cc:330] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:29.119787 7013 raft_consensus.cc:2229] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:29.120466 7013 raft_consensus.cc:2258] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:29.122169 7013 ts_tablet_manager.cc:1905] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:29.126848 6913 raft_consensus.cc:482] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:29.126925 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd6945ecccc4684a3fa5b2a1f07057f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:29.128444 6033 raft_consensus.cc:348] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.127368 6913 raft_consensus.cc:504] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.129136 6033 raft_consensus.cc:374] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:29.129379 6033 raft_consensus.cc:729] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:29.129936 6033 consensus_queue.cc:260] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.120332 7241 raft_consensus.cc:2781] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:29.140339 7241 raft_consensus.cc:482] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:29.140712 7241 raft_consensus.cc:3037] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:29.142025 6046 tablet_bootstrap.cc:654] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:29.122216 6901 leader_election.cc:288] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
W20260808 13:35:29.128881 5551 leader_election.cc:341] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:29.145073 6913 leader_election.cc:288] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:29.146427 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet e992a0294c404cfb9541b57782a05fc9 (table default.Table_6f4c6712b74748f58390a05ec941a26a [id=2b4fc1c958044df7939df70bc136d956]) successfully deleted
I20260808 13:35:29.147118 7014 tablet_replica.cc:330] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:29.148003 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fc9fdb5ef543da833f9af6ca5807a9" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:29.148402 7014 raft_consensus.cc:2229] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:29.148634 5634 raft_consensus.cc:2454] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:29.148881 7014 raft_consensus.cc:2258] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:29.150852 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_9936d69a1f7b4853a7c37d5e8da2077f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:29.152797 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9936d69a1f7b4853a7c37d5e8da2077f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:29.154217 6033 ts_tablet_manager.cc:1428] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.348s user 0.007s sys 0.000s
W20260808 13:35:29.159322 4832 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet adf9a03742a14cfe867c578c466838de consensus state has no leader
I20260808 13:35:29.160534 7014 ts_tablet_manager.cc:1905] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:29.165910 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet ddfd9e3f002445738df24ebb0a333440
I20260808 13:35:29.170369 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet ddfd9e3f002445738df24ebb0a333440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:29.171267 7282 tablet.cc:1722] T f26e054d1992452396d3f7334ba3602c P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:29.155961 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fc9fdb5ef543da833f9af6ca5807a9" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:29.175547 5881 raft_consensus.cc:2454] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:29.176978 6033 tablet_bootstrap.cc:492] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:29.182150 6033 tablet_bootstrap.cc:654] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:29.185576 7013 ts_tablet_manager.cc:1918] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:29.185959 7013 log.cc:1196] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/9260370cf8cb46a7b699581e4e8d46db
I20260808 13:35:29.186693 7013 ts_tablet_manager.cc:1939] T 9260370cf8cb46a7b699581e4e8d46db P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:29.189810 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 254) took 1466 ms. Trace:
I20260808 13:35:29.190012 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 31fc9fdb5ef543da833f9af6ca5807a9 (table default.Table_faaabef2a0ab42648427c33cda679fb7 [id=7afc114b9ab641df8c540f834036b70c]). Expected version 1 got 0
I20260808 13:35:29.190173 7013 rpcz_store.cc:276] 0808 13:35:27.723266 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.723401 (+ 135us) service_pool.cc:227] Handling call
0808 13:35:29.118931 (+1395530us) ts_tablet_manager.cc:1145] Deleting tablet 9260370cf8cb46a7b699581e4e8d46db
0808 13:35:29.118971 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.156876 (+ 37905us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.185333 (+ 28457us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.189740 (+ 4407us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1348052}
I20260808 13:35:29.190755 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:29.191315 7013 tablet_replica.cc:330] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:29.192071 7013 raft_consensus.cc:2229] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:29.224956 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_9d2867cdebf84440b7aa21a71faba9c4 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 2 got 1
I20260808 13:35:29.230584 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 293e9540498e420d8ec873472d0d29fc (DEFAULT_TABLE table=default.Table_93dd48e3ed114ba0a47115dac0dfcd7f [id=164d867c7a184825b03b3c1c8a280cf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.230944 5285 leader_election.cc:302] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:29.231791 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293e9540498e420d8ec873472d0d29fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.233709 7013 raft_consensus.cc:2258] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:29.235721 7013 ts_tablet_manager.cc:1905] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:29.243646 7310 raft_consensus.cc:482] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:29.245384 7310 raft_consensus.cc:504] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.248189 7310 leader_election.cc:288] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:29.244746 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet ddfd9e3f002445738df24ebb0a333440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:29.242832 6913 raft_consensus.cc:2781] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:29.250751 6913 raft_consensus.cc:482] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:29.251140 6913 raft_consensus.cc:3037] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:29.255550 5615 tablet_service.cc:1467] Processing CreateTablet for tablet d439b27c3ad34619972c71d652f1ff10 (DEFAULT_TABLE table=default.Table_6db6d1f1b6454336958e53566b8d48e0 [id=8217e191cfe74ee08e6f3c2840d87214]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:29.257324 5285 leader_election.cc:341] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 896389941179456d9d0ed3cd14726f11
I20260808 13:35:29.261438 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 9260370cf8cb46a7b699581e4e8d46db (table default.Table_4026b16c74ac4327ac4ba5f178bfcaf6 [id=b33430fc55db40dd8eb1c9e0396a093b]) successfully deleted
I20260808 13:35:29.263496 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d439b27c3ad34619972c71d652f1ff10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.263306 5334 tablet_service.cc:1467] Processing CreateTablet for tablet 293e9540498e420d8ec873472d0d29fc (DEFAULT_TABLE table=default.Table_93dd48e3ed114ba0a47115dac0dfcd7f [id=164d867c7a184825b03b3c1c8a280cf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.265090 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293e9540498e420d8ec873472d0d29fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.066466 7269 tablet.cc:1722] T c8f1618a1d044898bffbc82107e90840 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:29.275725 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_bbaf205810de458492573e8be388ece0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:29.278354 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bbaf205810de458492573e8be388ece0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:29.233381 7219 raft_consensus.cc:482] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5cd987c3dbdb4168beb73cd7e577c7cb)
I20260808 13:35:29.283813 7219 raft_consensus.cc:504] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.287503 7219 leader_election.cc:288] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:29.313493 6046 tablet_bootstrap.cc:492] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:29.169149 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet ddfd9e3f002445738df24ebb0a333440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:29.172168 7280 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.214s user 0.000s sys 0.003s
I20260808 13:35:29.317216 7199 ts_tablet_manager.cc:1918] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:29.317670 7199 log.cc:1196] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/024e7f2daaad4f48a79b94344ff35e98
I20260808 13:35:29.156607 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896389941179456d9d0ed3cd14726f11" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:29.318552 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:29.318929 5877 raft_consensus.cc:2454] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:29.319410 5630 raft_consensus.cc:2454] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:29.199877 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 293e9540498e420d8ec873472d0d29fc (DEFAULT_TABLE table=default.Table_93dd48e3ed114ba0a47115dac0dfcd7f [id=164d867c7a184825b03b3c1c8a280cf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.320461 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293e9540498e420d8ec873472d0d29fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.316597 6913 raft_consensus.cc:504] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.324275 6046 ts_tablet_manager.cc:1397] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.216s user 0.015s sys 0.037s
I20260808 13:35:29.330814 6913 leader_election.cc:288] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:29.208822 5772 consensus_peers.cc:487] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3682c9c656c4556bc13ae3aeddd7c5e. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:29.334977 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 126258us
W20260808 13:35:29.335772 5772 consensus_peers.cc:487] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e992a0294c404cfb9541b57782a05fc9. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:29.337210 5283 leader_election.cc:302] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:29.339130 6913 raft_consensus.cc:2781] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:29.339486 6913 raft_consensus.cc:482] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:29.339814 6913 raft_consensus.cc:3037] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:29.340197 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:29.345345 5285 leader_election.cc:302] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:29.346429 7309 raft_consensus.cc:2781] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:29.346958 7309 raft_consensus.cc:482] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:29.347314 7309 raft_consensus.cc:3037] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:29.347302 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fc9fdb5ef543da833f9af6ca5807a9" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:29.348464 5630 raft_consensus.cc:3037] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:29.165457 7241 raft_consensus.cc:504] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.351349 5877 raft_consensus.cc:2379] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:29.351521 6046 raft_consensus.cc:348] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.352342 6046 raft_consensus.cc:374] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:29.352628 6046 raft_consensus.cc:729] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:29.365949 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:29.229079 7246 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.644s user 0.002s sys 0.009s
I20260808 13:35:29.254599 5836 tablet_service.cc:1467] Processing CreateTablet for tablet d439b27c3ad34619972c71d652f1ff10 (DEFAULT_TABLE table=default.Table_6db6d1f1b6454336958e53566b8d48e0 [id=8217e191cfe74ee08e6f3c2840d87214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.367195 6046 consensus_queue.cc:260] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.370473 5336 tablet_service.cc:1467] Processing CreateTablet for tablet d439b27c3ad34619972c71d652f1ff10 (DEFAULT_TABLE table=default.Table_6db6d1f1b6454336958e53566b8d48e0 [id=8217e191cfe74ee08e6f3c2840d87214]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:29.374752 5285 leader_election.cc:341] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 06c59eea10ec45af8e3a0992f9cc74bf
I20260808 13:35:29.375308 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d439b27c3ad34619972c71d652f1ff10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.382050 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fc7cb305e35541c781c3e3790cf8a0af" } modify_external_catalogs: true
I20260808 13:35:29.382737 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fc7cb305e35541c781c3e3790cf8a0af" } modify_external_catalogs: true
I20260808 13:35:29.387154 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fd5435dd94cd493ab4780d4dea9de9a8" } new_table_name: "default.Table_e101c061f2bf4821b6fe2c6108432735" modify_external_catalogs: true
I20260808 13:35:29.393002 6046 ts_tablet_manager.cc:1428] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.068s user 0.004s sys 0.003s
I20260808 13:35:29.395658 6046 tablet_bootstrap.cc:492] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:29.406666 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d439b27c3ad34619972c71d652f1ff10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.406462 5834 tablet_service.cc:1467] Processing CreateTablet for tablet c1607745896141f68535b751dc211796 (DEFAULT_TABLE table=default.Table_c7297f7910b4405ab0483761e1aed004 [id=b0736fe025cb443ba8cbab989b9a4422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.408550 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1607745896141f68535b751dc211796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.410036 7014 ts_tablet_manager.cc:1918] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:29.413774 7014 log.cc:1196] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/024e7f2daaad4f48a79b94344ff35e98
I20260808 13:35:29.420230 7014 ts_tablet_manager.cc:1939] T 024e7f2daaad4f48a79b94344ff35e98 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:29.424156 7219 raft_consensus.cc:482] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:29.427070 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_94d7e54302ae4463a43a530d34d4f264"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:29.424930 7219 raft_consensus.cc:504] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.427815 7261 consensus_queue.cc:1035] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:35:29.428889 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94d7e54302ae4463a43a530d34d4f264 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:29.436916 5283 leader_election.cc:341] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: 06c59eea10ec45af8e3a0992f9cc74bf
I20260808 13:35:29.437410 5283 leader_election.cc:302] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:29.438033 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:29.438328 7211 raft_consensus.cc:2726] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:29.438925 5636 raft_consensus.cc:3037] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:29.441637 7219 leader_election.cc:288] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:29.446328 5607 tablet_service.cc:1467] Processing CreateTablet for tablet c1607745896141f68535b751dc211796 (DEFAULT_TABLE table=default.Table_c7297f7910b4405ab0483761e1aed004 [id=b0736fe025cb443ba8cbab989b9a4422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.447340 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1607745896141f68535b751dc211796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.318509 7199 ts_tablet_manager.cc:1939] T 024e7f2daaad4f48a79b94344ff35e98 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:29.446380 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 239) took 2318 ms. Trace:
I20260808 13:35:29.455267 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f2e0b98af431d99894e8778257bc4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:29.422250 7013 ts_tablet_manager.cc:1918] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:29.461702 7013 log.cc:1196] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/f0c76b819474468baac51365360becfc
I20260808 13:35:29.467271 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fc9fdb5ef543da833f9af6ca5807a9" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:29.468583 5877 raft_consensus.cc:3037] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:29.473569 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 3451d72548a140c68a6ccc2b566a09b1 (DEFAULT_TABLE table=default.Table_6529b65492c94717b809b0a0c5f23509 [id=90cb51eadbd94240966b813b54f4b704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.474861 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3451d72548a140c68a6ccc2b566a09b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.486930 7013 ts_tablet_manager.cc:1939] T f0c76b819474468baac51365360becfc P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:29.471388 7282 tablet.cc:1722] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:29.366487 7241 leader_election.cc:288] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:29.496836 5773 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.727s user 0.000s sys 0.014s
I20260808 13:35:29.510119 5334 tablet_service.cc:1467] Processing CreateTablet for tablet c1607745896141f68535b751dc211796 (DEFAULT_TABLE table=default.Table_c7297f7910b4405ab0483761e1aed004 [id=b0736fe025cb443ba8cbab989b9a4422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.512094 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1607745896141f68535b751dc211796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.527436 7192 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:46328 (request call id 266) took 1298 ms. Trace:
I20260808 13:35:29.532856 5384 raft_consensus.cc:3037] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:29.536873 5345 tablet_service.cc:1467] Processing CreateTablet for tablet 3451d72548a140c68a6ccc2b566a09b1 (DEFAULT_TABLE table=default.Table_6529b65492c94717b809b0a0c5f23509 [id=90cb51eadbd94240966b813b54f4b704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.531358 7327 tablet.cc:1722] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:29.545382 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab26fa1872e4acf874f6577bded5e43 (DEFAULT_TABLE table=default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:29.497394 5883 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s user 0.005s sys 0.006s
W20260808 13:35:29.554980 5883 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s user 0.005s sys 0.006s
W20260808 13:35:29.556208 5773 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.788s user 0.000s sys 0.015s
I20260808 13:35:29.557636 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127e8d0323954ef788018bcb91582951" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:29.514895 7281 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:54830 (request call id 254) took 1775 ms. Trace:
I20260808 13:35:29.545118 6913 raft_consensus.cc:504] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.524909 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f2e0b98af431d99894e8778257bc4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:29.530058 7240 raft_consensus.cc:482] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5cd987c3dbdb4168beb73cd7e577c7cb)
I20260808 13:35:29.570052 7240 raft_consensus.cc:504] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:29.597186 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3451d72548a140c68a6ccc2b566a09b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.597997 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab26fa1872e4acf874f6577bded5e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.598544 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 253) took 1901 ms. Trace:
I20260808 13:35:29.598912 7013 rpcz_store.cc:276] 0808 13:35:27.696927 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.722315 (+ 25388us) service_pool.cc:227] Handling call
0808 13:35:29.191174 (+1468859us) ts_tablet_manager.cc:1145] Deleting tablet f0c76b819474468baac51365360becfc
0808 13:35:29.191215 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.315841 (+124626us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.422027 (+106186us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.598493 (+176466us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":41109,"tablet-delete.queue_time_us":1420108}
I20260808 13:35:29.600253 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 252) took 1903 ms. Trace:
I20260808 13:35:29.600544 7013 rpcz_store.cc:276] 0808 13:35:27.696648 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.726329 (+ 29681us) service_pool.cc:227] Handling call
0808 13:35:29.599999 (+1873670us) ts_tablet_manager.cc:1145] Deleting tablet f0c76b819474468baac51365360becfc
0808 13:35:29.600042 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.600207 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1827012}
I20260808 13:35:29.601439 7013 tablet_replica.cc:330] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:29.604451 7240 leader_election.cc:288] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:29.449335 7014 rpcz_store.cc:276] 0808 13:35:27.127867 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.128321 (+ 454us) service_pool.cc:227] Handling call
0808 13:35:29.145254 (+2016933us) ts_tablet_manager.cc:1145] Deleting tablet 024e7f2daaad4f48a79b94344ff35e98
0808 13:35:29.145314 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.231452 (+ 86138us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.408277 (+176825us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.445511 (+ 37234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2015914}
I20260808 13:35:29.605690 5637 raft_consensus.cc:2454] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:29.612059 6026 tablet_bootstrap.cc:654] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:29.594821 5283 leader_election.cc:341] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:29.613456 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab26fa1872e4acf874f6577bded5e43 (DEFAULT_TABLE table=default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.614507 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab26fa1872e4acf874f6577bded5e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.615175 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet f0c76b819474468baac51365360becfc (table default.Table_c4e53d8c393449429ef2791497ee7208 [id=26fb789fff434bb0825daf68d5378013]) successfully deleted
W20260808 13:35:29.615895 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet f0c76b819474468baac51365360becfc because the tablet was not found. No further retry: Not found: Tablet not found: f0c76b819474468baac51365360becfc
I20260808 13:35:29.620787 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 3451d72548a140c68a6ccc2b566a09b1 (DEFAULT_TABLE table=default.Table_6529b65492c94717b809b0a0c5f23509 [id=90cb51eadbd94240966b813b54f4b704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.621981 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3451d72548a140c68a6ccc2b566a09b1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:29.624468 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 538d9a883e9d46668fde30876352d20e consensus state has no leader
I20260808 13:35:29.626533 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab26fa1872e4acf874f6577bded5e43 (DEFAULT_TABLE table=default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:29.627588 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab26fa1872e4acf874f6577bded5e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:29.566998 7281 rpcz_store.cc:276] 0808 13:35:27.739086 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.739368 (+ 282us) service_pool.cc:227] Handling call
0808 13:35:29.514807 (+1775439us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.741161 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:27.741806 (+ 645us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:27.741844 (+ 38us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:27.742095 (+ 251us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:27.742316 (+ 221us) alter_schema_op.cc:126] START. Timestamp: 1786196127742142
0808 13:35:27.742330 (+ 14us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.743527 (+ 1197us) log.cc:842] Serialized 173 byte log entry
0808 13:35:28.132642 (+389115us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b5400010f20. stack: 00007f19df247033 00007f19df246f30 00007f19dedc8534 00000000004db2b4 00000000004db249 00007f19e6153471 00007f19e615c771 00007f19e6167220 00007f19e61671c8 00007f19e6167148 00007f19e6167103 00007f19e616638c 00007f19e8151206 00007f19e8149808 00007f19e61a338b 00007f19e61a5e80
0808 13:35:28.558282 (+425640us) op_driver.cc:444] REPLICATION: finished
0808 13:35:28.923865 (+365583us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:28.990413 (+ 66548us) log.cc:842] Serialized 5 byte log entry
0808 13:35:29.198370 (+207957us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.481093 (+282723us) log.cc:842] Serialized 14 byte log entry
0808 13:35:29.482171 (+ 1078us) alter_schema_op.cc:71] Released schema lock
0808 13:35:29.482267 (+ 96us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":248815,"prepare.queue_time_us":351,"prepare.run_cpu_time_us":82733,"prepare.run_wall_time_us":419595,"replication_time_us":815833,"spinlock_wait_cycles":95609216,"thread_start_us":398684,"threads_started":2,"wal-append.queue_time_us":344668}]]}
I20260808 13:35:29.628237 6046 tablet_bootstrap.cc:654] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:29.629343 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f2e0b98af431d99894e8778257bc4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:29.634729 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:29.635437 5875 raft_consensus.cc:2379] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:29.636708 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 024e7f2daaad4f48a79b94344ff35e98 (table default.Table_fad1a84e78af4cfe8c7ceee0e14a75e5 [id=cda1418c2bf5405abda61222d65589cf]) successfully deleted
I20260808 13:35:29.641151 4835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 0756b5f24ed94a9fbe2a43efc7b8d40d
I20260808 13:35:29.642529 4835 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 311) took 1017 ms. Trace:
I20260808 13:35:29.643052 5845 tablet_service.cc:1514] Processing DeleteTablet for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:29.581018 7280 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:54830 (request call id 248) took 2101 ms. Trace:
I20260808 13:35:29.642835 4835 rpcz_store.cc:276] 0808 13:35:28.624804 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.681386 (+ 56582us) service_pool.cc:227] Handling call
0808 13:35:28.687997 (+ 6611us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:28.688584 (+ 587us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:28.688703 (+ 119us) catalog_manager.cc:2761] Locking tablets
0808 13:35:29.184028 (+495325us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:29.640752 (+456724us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:29.640977 (+ 225us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:29.641072 (+ 95us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:29.642479 (+ 1407us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:29.478392 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:29.478813 (+ 421us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:29.478838 (+ 25us) write_op.cc:435] Acquired schema lock
0808 13:35:29.478851 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:29.479172 (+ 321us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:29.479208 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:29.479230 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:29.479617 (+ 387us) tablet.cc:699] Row locks acquired
0808 13:35:29.479637 (+ 20us) write_op.cc:260] PREPARE: finished
0808 13:35:29.479781 (+ 144us) write_op.cc:270] Start()
0808 13:35:29.479900 (+ 119us) write_op.cc:276] Timestamp: P: 1786196129479701 usec, L: 0
0808 13:35:29.479919 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:29.480589 (+ 670us) log.cc:842] Serialized 725 byte log entry
0808 13:35:29.639169 (+158580us) op_driver.cc:444] REPLICATION: finished
0808 13:35:29.639423 (+ 254us) write_op.cc:301] APPLY: starting
0808 13:35:29.639484 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:29.639595 (+ 111us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:29.639699 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:29.639940 (+ 241us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:29.640067 (+ 127us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:29.640107 (+ 40us) write_op.cc:312] APPLY: finished
0808 13:35:29.640296 (+ 189us) log.cc:842] Serialized 29 byte log entry
0808 13:35:29.640437 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:29.640498 (+ 61us) write_op.cc:454] Released schema lock
0808 13:35:29.640537 (+ 39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":96,"num_ops":2,"prepare.queue_time_us":292668,"prepare.run_cpu_time_us":2538,"prepare.run_wall_time_us":76087,"raft.queue_time_us":3634,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":15,"replication_time_us":159161}]]}
I20260808 13:35:29.646001 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf907a9f19f4d64b27be946011a8ddb" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:29.595417 7334 tablet.cc:1722] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:29.596765 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 243) took 2359 ms. Trace:
I20260808 13:35:29.649055 7199 rpcz_store.cc:276] 0808 13:35:27.237455 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.239786 (+ 2331us) service_pool.cc:227] Handling call
0808 13:35:28.139085 (+899299us) ts_tablet_manager.cc:1145] Deleting tablet 024e7f2daaad4f48a79b94344ff35e98
0808 13:35:28.139131 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:28.770316 (+631185us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.316942 (+546626us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.596708 (+279766us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":898800}
I20260808 13:35:29.650384 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 024e7f2daaad4f48a79b94344ff35e98 (table default.Table_fad1a84e78af4cfe8c7ceee0e14a75e5 [id=cda1418c2bf5405abda61222d65589cf]) successfully deleted
I20260808 13:35:29.652518 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 240) took 2441 ms. Trace:
I20260808 13:35:29.652868 7199 rpcz_store.cc:276] 0808 13:35:27.220318 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.228864 (+ 8546us) service_pool.cc:227] Handling call
0808 13:35:29.652253 (+2423389us) ts_tablet_manager.cc:1145] Deleting tablet 2173e97590be43f9a24f55689d796269
0808 13:35:29.652297 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.652465 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2381499}
W20260808 13:35:29.654007 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 2173e97590be43f9a24f55689d796269 because the tablet was not found. No further retry: Not found: Tablet not found: 2173e97590be43f9a24f55689d796269
I20260808 13:35:29.657994 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 238) took 2530 ms. Trace:
I20260808 13:35:29.658315 7014 rpcz_store.cc:276] 0808 13:35:27.127596 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.146712 (+ 19116us) service_pool.cc:227] Handling call
0808 13:35:29.607099 (+2460387us) ts_tablet_manager.cc:1145] Deleting tablet 2cd2d0c9ab2c4704a74d06559bb913d6
0808 13:35:29.638568 (+ 31469us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.657928 (+ 19360us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2459728}
I20260808 13:35:29.660380 7014 tablet_replica.cc:330] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:29.661126 7014 raft_consensus.cc:2229] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:29.661556 7014 raft_consensus.cc:2258] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:29.659708 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4dc66771804e0cb8991da63eacba0c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:29.664089 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:29.664242 5285 leader_election.cc:302] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:29.665107 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 because the tablet was not found. No further retry: Not found: Tablet not found: 2cd2d0c9ab2c4704a74d06559bb913d6
I20260808 13:35:29.667125 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5398341b051f491ca73c85f9c8332eb9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:29.668607 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5398341b051f491ca73c85f9c8332eb9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:29.694574 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fc9fdb5ef543da833f9af6ca5807a9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:29.694904 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a384cb98038948a2aa2ea653c22ac2eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:29.695611 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd6945ecccc4684a3fa5b2a1f07057f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:29.696476 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a384cb98038948a2aa2ea653c22ac2eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:29.696740 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 235) took 2524 ms. Trace:
I20260808 13:35:29.697139 7199 rpcz_store.cc:276] 0808 13:35:27.172108 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.184567 (+ 12459us) service_pool.cc:227] Handling call
0808 13:35:29.688466 (+2503899us) ts_tablet_manager.cc:1145] Deleting tablet 973a7afdcbf944aab8f11b3fd3995865
0808 13:35:29.694408 (+ 5942us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.696681 (+ 2273us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2391154}
I20260808 13:35:29.699361 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 246) took 2286 ms. Trace:
I20260808 13:35:29.644856 7280 rpcz_store.cc:276] 0808 13:35:27.480065 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.480201 (+ 136us) service_pool.cc:227] Handling call
0808 13:35:29.580951 (+2100750us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.513383 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:27.513724 (+ 341us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:27.513753 (+ 29us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:27.513803 (+ 50us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:27.513912 (+ 109us) alter_schema_op.cc:126] START. Timestamp: 1786196127513845
0808 13:35:27.513930 (+ 18us) op_driver.cc:331] REPLICATION: starting
0808 13:35:27.514487 (+ 557us) log.cc:842] Serialized 173 byte log entry
0808 13:35:28.588899 (+1074412us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b180000d2a0. stack: 00007f19df247033 00007f19df246f30 00007f19dedc8534 00000000004db2b4 00000000004db249 00007f19e80bf8b1 00007f19e80cc661 00007f19e662c8a8 00007f19e662ea78 00007f19e662ea28 00007f19e662e9a8 00007f19e662e963 00007f19e662dc4c 00007f19e8151206 00007f19e8149808 00007f19e61da170
0808 13:35:28.588933 (+ 34us) op_driver.cc:444] REPLICATION: finished
0808 13:35:28.956184 (+367251us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:28.958456 (+ 2272us) log.cc:842] Serialized 5 byte log entry
0808 13:35:29.579701 (+621245us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.580691 (+ 990us) log.cc:842] Serialized 14 byte log entry
0808 13:35:29.580797 (+ 106us) alter_schema_op.cc:71] Released schema lock
0808 13:35:29.580824 (+ 27us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":366981,"prepare.queue_time_us":32351,"prepare.run_cpu_time_us":2862,"prepare.run_wall_time_us":62614,"raft.queue_time_us":628,"raft.run_cpu_time_us":815,"raft.run_wall_time_us":814,"replication_time_us":1061029,"spinlock_wait_cycles":5938048,"thread_start_us":495697,"threads_started":4,"wal-append.queue_time_us":259847,"wal-append.run_cpu_time_us":1389,"wal-append.run_wall_time_us":1001415}]]}
I20260808 13:35:29.659665 7309 raft_consensus.cc:504] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.721377 5386 raft_consensus.cc:2379] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:29.720605 6033 tablet_bootstrap.cc:492] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:29.721869 6033 ts_tablet_manager.cc:1397] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.545s user 0.010s sys 0.008s
I20260808 13:35:29.722267 7309 leader_election.cc:288] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:29.723033 6913 leader_election.cc:288] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:29.724169 6033 raft_consensus.cc:348] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.724807 6033 raft_consensus.cc:374] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:29.725062 6033 raft_consensus.cc:729] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:29.725950 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:29.699688 7199 rpcz_store.cc:276] 0808 13:35:27.412904 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.413048 (+ 144us) service_pool.cc:227] Handling call
0808 13:35:29.699027 (+2285979us) ts_tablet_manager.cc:1145] Deleting tablet daa59a18ae624bc8975bee2c460895a7
0808 13:35:29.699075 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.699312 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2285441}
I20260808 13:35:29.732610 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 247) took 2254 ms. Trace:
I20260808 13:35:29.732936 7199 rpcz_store.cc:276] 0808 13:35:27.477827 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.478016 (+ 189us) service_pool.cc:227] Handling call
0808 13:35:29.732238 (+2254222us) ts_tablet_manager.cc:1145] Deleting tablet b314734698ed423ba9cf4134db5846ec
0808 13:35:29.732290 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.732556 (+ 266us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2253525}
I20260808 13:35:29.742894 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 257) took 1715 ms. Trace:
I20260808 13:35:29.743321 7199 rpcz_store.cc:276] 0808 13:35:28.027781 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.027942 (+ 161us) service_pool.cc:227] Handling call
0808 13:35:29.742496 (+1714554us) ts_tablet_manager.cc:1145] Deleting tablet 45a26598a69146e1950e9227523fff13
0808 13:35:29.742543 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.742825 (+ 282us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1708482}
I20260808 13:35:29.531204 7192 rpcz_store.cc:276] 0808 13:35:28.229161 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.229261 (+ 100us) service_pool.cc:227] Handling call
0808 13:35:29.526686 (+1297425us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:28.250997 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:28.251310 (+ 313us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:28.251337 (+ 27us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:28.251403 (+ 66us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:28.251507 (+ 104us) alter_schema_op.cc:126] START. Timestamp: 1786196128251417
0808 13:35:28.251526 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:28.251966 (+ 440us) log.cc:842] Serialized 173 byte log entry
0808 13:35:28.605564 (+353598us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b5400045c98. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c27a00605 00007f5c27a00752 00007f5c279bcb94 00007f5c279a5545 00007f5c279aa771 00007f5c279aa6f9 00007f5c279aa689 00007f5c279aa651 00007f5c279a981d 0000000000580784
0808 13:35:28.936321 (+330757us) op_driver.cc:444] REPLICATION: finished
0808 13:35:28.936972 (+ 651us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:28.947530 (+ 10558us) log.cc:842] Serialized 5 byte log entry
0808 13:35:29.394800 (+447270us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.502282 (+107482us) log.cc:842] Serialized 14 byte log entry
0808 13:35:29.503088 (+ 806us) alter_schema_op.cc:71] Released schema lock
0808 13:35:29.504140 (+ 1052us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":266,"prepare.queue_time_us":4233,"prepare.run_cpu_time_us":13365,"prepare.run_wall_time_us":350052,"raft.queue_time_us":25520,"raft.run_cpu_time_us":6209,"raft.run_wall_time_us":41480,"replication_time_us":684713,"spinlock_wait_cycles":4286336,"thread_start_us":306876,"threads_started":1,"wal-append.queue_time_us":298271}]]}
W20260808 13:35:29.731724 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet daa59a18ae624bc8975bee2c460895a7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet daa59a18ae624bc8975bee2c460895a7 already in progress: creating tablet
W20260808 13:35:29.750161 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet b314734698ed423ba9cf4134db5846ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b314734698ed423ba9cf4134db5846ec already in progress: creating tablet
W20260808 13:35:29.750847 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 45a26598a69146e1950e9227523fff13 because the tablet was not found. No further retry: Not found: Tablet not found: 45a26598a69146e1950e9227523fff13
I20260808 13:35:29.755069 7199 tablet_replica.cc:330] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:29.755720 7219 raft_consensus.cc:2781] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:29.759912 7211 raft_consensus.cc:482] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:29.795666 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4dc66771804e0cb8991da63eacba0c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:29.776620 7219 raft_consensus.cc:482] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:29.762267 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896389941179456d9d0ed3cd14726f11" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:29.801066 5878 raft_consensus.cc:3037] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:29.763288 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:29.779870 7240 consensus_queue.cc:1035] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:29.798177 5877 raft_consensus.cc:2454] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:29.807641 5875 raft_consensus.cc:2379] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:29.818526 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 31fc9fdb5ef543da833f9af6ca5807a9
I20260808 13:35:29.820222 4833 catalog_manager.cc:6565] ReplaceTablet: tablet 31fc9fdb5ef543da833f9af6ca5807a9 deleted and replaced by tablet 67327badb2524a2c817045a893bcad6e
I20260808 13:35:29.824687 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 31fc9fdb5ef543da833f9af6ca5807a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 67327badb2524a2c817045a893bcad6e) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:29.830081 5607 tablet_service.cc:1514] Processing DeleteTablet for tablet 31fc9fdb5ef543da833f9af6ca5807a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 67327badb2524a2c817045a893bcad6e) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:29.790269 7327 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:46328 (request call id 277) took 1126 ms. Trace:
I20260808 13:35:29.838210 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet 31fc9fdb5ef543da833f9af6ca5807a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 67327badb2524a2c817045a893bcad6e) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:29.838202 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 174011us
I20260808 13:35:29.846685 5878 raft_consensus.cc:2454] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:29.824223 7219 raft_consensus.cc:3037] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:29.853817 6033 consensus_queue.cc:260] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.791285 7211 raft_consensus.cc:504] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.861577 6033 ts_tablet_manager.cc:1428] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.139s user 0.006s sys 0.000s
I20260808 13:35:29.862624 6033 tablet_bootstrap.cc:492] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:29.871187 7211 leader_election.cc:288] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:29.886497 4784 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 89734.4us
I20260808 13:35:29.835600 7327 rpcz_store.cc:276] 0808 13:35:28.660599 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.660769 (+ 170us) service_pool.cc:227] Handling call
0808 13:35:29.786670 (+1125901us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:28.686717 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:28.687338 (+ 621us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:28.687396 (+ 58us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:28.687644 (+ 248us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:28.687844 (+ 200us) alter_schema_op.cc:126] START. Timestamp: 1786196128687703
0808 13:35:28.687865 (+ 21us) op_driver.cc:331] REPLICATION: starting
0808 13:35:28.688721 (+ 856us) log.cc:842] Serialized 173 byte log entry
0808 13:35:29.461644 (+772923us) op_driver.cc:444] REPLICATION: finished
0808 13:35:29.525927 (+ 64283us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:29.543345 (+ 17418us) log.cc:842] Serialized 5 byte log entry
0808 13:35:29.717432 (+174087us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:29.719436 (+ 2004us) log.cc:842] Serialized 14 byte log entry
0808 13:35:29.725552 (+ 6116us) alter_schema_op.cc:71] Released schema lock
0808 13:35:29.744854 (+ 19302us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":63811,"prepare.queue_time_us":511,"prepare.run_cpu_time_us":11278,"prepare.run_wall_time_us":97869,"raft.queue_time_us":177,"raft.run_cpu_time_us":1531,"raft.run_wall_time_us":1533,"replication_time_us":773589,"thread_start_us":157235,"threads_started":2,"wal-append.queue_time_us":63976}]]}
I20260808 13:35:29.835640 7013 raft_consensus.cc:2229] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:29.917021 7013 raft_consensus.cc:2258] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:29.919127 7013 ts_tablet_manager.cc:1905] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:29.925374 6033 tablet_bootstrap.cc:654] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:29.957305 6033 tablet_bootstrap.cc:492] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:29.958004 6033 ts_tablet_manager.cc:1397] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.096s user 0.010s sys 0.006s
I20260808 13:35:29.962150 6033 raft_consensus.cc:348] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.964557 6033 raft_consensus.cc:374] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:29.964879 6033 raft_consensus.cc:729] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:29.972126 7014 ts_tablet_manager.cc:1905] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:29.975587 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a73e98a8b20144508a5a8d451f4bb302 consensus state has no leader
I20260808 13:35:29.977361 6033 consensus_queue.cc:260] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:29.980141 6033 ts_tablet_manager.cc:1428] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.022s user 0.008s sys 0.000s
I20260808 13:35:29.995740 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261e45848f424a349b0c5290954c5bc6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
W20260808 13:35:30.007004 5550 leader_election.cc:341] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.007524 5550 leader_election.cc:302] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:30.039000 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 228482us
I20260808 13:35:30.052785 5772 leader_election.cc:302] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:30.054571 7241 raft_consensus.cc:2726] T cc4dc66771804e0cb8991da63eacba0c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:30.061309 6026 tablet_bootstrap.cc:492] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:30.061777 6026 ts_tablet_manager.cc:1397] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 1.015s user 0.015s sys 0.000s
W20260808 13:35:30.064289 5285 leader_election.cc:341] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 896389941179456d9d0ed3cd14726f11
W20260808 13:35:30.065557 5285 leader_election.cc:341] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: e992a0294c404cfb9541b57782a05fc9
I20260808 13:35:29.815492 5283 leader_election.cc:302] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:30.099834 7309 raft_consensus.cc:2781] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:30.100471 7309 raft_consensus.cc:686] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:30.101457 7309 consensus_queue.cc:237] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:30.108559 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 293169us
I20260808 13:35:30.109272 5283 leader_election.cc:302] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.110390 5283 leader_election.cc:302] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:29.946534 7199 raft_consensus.cc:2229] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:30.112622 7199 raft_consensus.cc:2258] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:30.122210 4823 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c -> Peer 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135): Couldn't send request to peer 87e8b888041b4135af55792050defcb5. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260808 13:35:30.131723 6901 raft_consensus.cc:2726] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:30.132834 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 73691.3us
W20260808 13:35:30.133642 5636 consensus_meta.cc:287] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.694s user 0.006s sys 0.006s
I20260808 13:35:30.133952 5636 raft_consensus.cc:2454] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:30.135272 6026 raft_consensus.cc:348] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.135951 6026 raft_consensus.cc:374] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:30.136247 6026 raft_consensus.cc:729] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:30.136866 6026 consensus_queue.cc:260] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.139958 5772 leader_election.cc:302] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:30.141350 7334 tablet.cc:1722] T 344828b16cfc47d5b419f8b569055fa4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:30.143990 7240 raft_consensus.cc:2781] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:30.144500 7240 raft_consensus.cc:686] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:30.145330 7240 consensus_queue.cc:237] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.149924 6033 tablet_bootstrap.cc:492] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:30.156642 6033 tablet_bootstrap.cc:654] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:30.159317 7346 raft_consensus.cc:2726] T 261e45848f424a349b0c5290954c5bc6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 13:35:30.159917 7211 raft_consensus.cc:2781] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:30.160429 7211 raft_consensus.cc:686] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:30.161242 7211 consensus_queue.cc:237] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.165338 6046 tablet_bootstrap.cc:492] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:30.166317 6046 ts_tablet_manager.cc:1397] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.771s user 0.014s sys 0.010s
I20260808 13:35:30.172744 6026 ts_tablet_manager.cc:1428] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.111s user 0.007s sys 0.000s
I20260808 13:35:30.171610 7334 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:54830 (request call id 253) took 2437 ms. Trace:
I20260808 13:35:30.167841 7199 ts_tablet_manager.cc:1905] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:30.189263 6033 tablet_bootstrap.cc:492] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:30.189960 6033 ts_tablet_manager.cc:1397] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.040s user 0.012s sys 0.001s
I20260808 13:35:30.175258 7334 rpcz_store.cc:276] 0808 13:35:27.734164 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.734320 (+ 156us) service_pool.cc:227] Handling call
0808 13:35:30.171534 (+2437214us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:27.735733 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:27.736176 (+ 443us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:29.581462 (+1845286us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:29.581535 (+ 73us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:29.581675 (+ 140us) alter_schema_op.cc:126] START. Timestamp: 1786196129581551
0808 13:35:29.581708 (+ 33us) op_driver.cc:331] REPLICATION: starting
0808 13:35:29.582216 (+ 508us) log.cc:842] Serialized 173 byte log entry
0808 13:35:30.140781 (+558565us) op_driver.cc:444] REPLICATION: finished
0808 13:35:30.141247 (+ 466us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:30.142893 (+ 1646us) log.cc:842] Serialized 5 byte log entry
0808 13:35:30.168298 (+ 25405us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:30.170574 (+ 2276us) log.cc:842] Serialized 14 byte log entry
0808 13:35:30.171359 (+ 785us) alter_schema_op.cc:71] Released schema lock
0808 13:35:30.171396 (+ 37us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"spinlock_wait_cycles":16256,"child_traces":[["op",{"apply.queue_time_us":251,"mutex_wait_us":14631,"prepare.queue_time_us":209,"prepare.run_cpu_time_us":43936,"prepare.run_wall_time_us":1861501,"raft.queue_time_us":21187,"raft.run_cpu_time_us":1402,"raft.run_wall_time_us":1992,"replication_time_us":558968}]]}
I20260808 13:35:30.195906 6033 raft_consensus.cc:348] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.196477 7219 raft_consensus.cc:504] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:30.197103 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 86753.9us
I20260808 13:35:30.198253 6033 raft_consensus.cc:374] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:30.198678 6033 raft_consensus.cc:729] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:30.202348 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e992a0294c404cfb9541b57782a05fc9" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:30.202044 7334 tablet.cc:1722] T c8f1618a1d044898bffbc82107e90840 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:30.205053 7240 raft_consensus.cc:482] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:30.205755 7240 raft_consensus.cc:504] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.204397 6033 consensus_queue.cc:260] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.209857 7240 leader_election.cc:288] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:30.211690 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234591e698f8472e90e1f610da37ff3a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:30.213594 5771 leader_election.cc:341] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.211236 6033 ts_tablet_manager.cc:1428] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.021s user 0.003s sys 0.004s
I20260808 13:35:30.215078 4832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d30a1f7564df4e6085a1858c6f3924fc" } new_table_name: "default.Table_f1ee5ef8d2d34ba8844275610eb0a394" modify_external_catalogs: true
I20260808 13:35:30.218801 4836 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a] alter complete (version 2)
I20260808 13:35:30.222496 5283 leader_election.cc:302] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.223235 7346 raft_consensus.cc:2726] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:30.227301 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:25 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:30.228978 4836 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 43) took 1019 ms (client timeout 1000 ms). Trace:
I20260808 13:35:30.246835 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet 973a7afdcbf944aab8f11b3fd3995865 with delete_type TABLET_DATA_DELETED (replaced by tablet 3424ca753bcc4f448d314c56aba3323e) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:30.248618 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fb3e099f7f4544b3c100070f0d14ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:30.250597 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a26598a69146e1950e9227523fff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:30.251750 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e with delete_type TABLET_DATA_DELETED (replaced by tablet ba39706cff65404485959865fff1772a) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:30.253880 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c0f49fa8313743fbb5ea5d76f5262610" } modify_external_catalogs: true
I20260808 13:35:30.254451 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c0f49fa8313743fbb5ea5d76f5262610" } modify_external_catalogs: true
W20260808 13:35:30.229318 4836 rpcz_store.cc:269] 0808 13:35:29.209269 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:29.209397 (+ 128us) service_pool.cc:227] Handling call
0808 13:35:30.228901 (+1019504us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:29.559179 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:29.615216 (+ 56037us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:29.615240 (+ 24us) write_op.cc:435] Acquired schema lock
0808 13:35:29.615253 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:29.615551 (+ 298us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:29.615591 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:29.615615 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:29.615757 (+ 142us) tablet.cc:699] Row locks acquired
0808 13:35:29.615770 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:29.615885 (+ 115us) write_op.cc:270] Start()
0808 13:35:29.615969 (+ 84us) write_op.cc:276] Timestamp: P: 1786196129615805 usec, L: 0
0808 13:35:29.615984 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:29.652625 (+ 36641us) log.cc:842] Serialized 356 byte log entry
0808 13:35:29.653262 (+ 637us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:30.215145 (+561883us) op_driver.cc:444] REPLICATION: finished
0808 13:35:30.215473 (+ 328us) write_op.cc:301] APPLY: starting
0808 13:35:30.215561 (+ 88us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:30.215780 (+ 219us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:30.215830 (+ 50us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:30.216142 (+ 312us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:30.216450 (+ 308us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:30.216558 (+ 108us) write_op.cc:312] APPLY: finished
0808 13:35:30.216864 (+ 306us) log.cc:842] Serialized 23 byte log entry
0808 13:35:30.217135 (+ 271us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:30.217262 (+ 127us) write_op.cc:454] Released schema lock
0808 13:35:30.217302 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:30.217605 (+ 303us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:30.217681 (+ 76us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":120,"apply.run_cpu_time_us":2453,"apply.run_wall_time_us":2453,"num_ops":1,"prepare.queue_time_us":330905,"prepare.run_cpu_time_us":3305,"prepare.run_wall_time_us":94367,"raft.queue_time_us":2752,"raft.run_cpu_time_us":657,"raft.run_wall_time_us":656,"replication_time_us":599087,"spinlock_wait_cycles":83126016}]]}
I20260808 13:35:30.254750 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c76b819474468baac51365360becfc with delete_type TABLET_DATA_DELETED (replaced by tablet 4b7284afe0294644a368cf51451c7ab6) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:30.255931 4836 master_service.cc:946] ReplaceTablet: received request to replace tablet 1f9bc338093f403881d329a7e913c9f7 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:30.256680 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet 9260370cf8cb46a7b699581e4e8d46db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:30.272233 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:12 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:30.293390 6046 raft_consensus.cc:348] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.294122 6046 raft_consensus.cc:374] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:30.296612 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_9d2867cdebf84440b7aa21a71faba9c4 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 2 got 1
I20260808 13:35:30.297509 6026 tablet_bootstrap.cc:492] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:30.303957 7014 ts_tablet_manager.cc:1918] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:30.304529 7014 log.cc:1196] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/a541ec35b1a44109866fabc953f14cc6
I20260808 13:35:30.305326 7014 ts_tablet_manager.cc:1939] T a541ec35b1a44109866fabc953f14cc6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
W20260808 13:35:30.308849 5630 consensus_meta.cc:287] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.960s user 0.007s sys 0.003s
I20260808 13:35:30.309520 6033 tablet_bootstrap.cc:492] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:30.309229 5630 raft_consensus.cc:2454] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
W20260808 13:35:30.311084 4838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 49) took 1686 ms (client timeout 1000 ms). Trace:
W20260808 13:35:30.322890 4838 rpcz_store.cc:269] 0808 13:35:28.624394 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.624928 (+ 534us) service_pool.cc:227] Handling call
0808 13:35:28.699682 (+ 74754us) spinlock_profiling.cc:243] Waited 69.1 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:30.311004 (+1611322us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":28,"spinlock_wait_cycles":159018368}
I20260808 13:35:30.327004 7219 leader_election.cc:288] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:30.294426 6046 raft_consensus.cc:729] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:30.334554 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f2e0b98af431d99894e8778257bc4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:30.334893 6046 consensus_queue.cc:260] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.339721 6046 ts_tablet_manager.cc:1428] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.170s user 0.007s sys 0.000s
W20260808 13:35:30.345141 5283 leader_election.cc:341] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.345623 6033 tablet_bootstrap.cc:654] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:30.347189 4841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet ba39706cff65404485959865fff1772a
I20260808 13:35:30.350157 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet ba39706cff65404485959865fff1772a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:30.351341 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet ba39706cff65404485959865fff1772a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:30.352347 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet ba39706cff65404485959865fff1772a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:30.328006 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f2e0b98af431d99894e8778257bc4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
W20260808 13:35:30.372879 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 178176us
I20260808 13:35:30.374593 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234591e698f8472e90e1f610da37ff3a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:30.374884 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 241) took 3180 ms. Trace:
I20260808 13:35:30.375250 5640 raft_consensus.cc:2454] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:30.375172 7014 rpcz_store.cc:276] 0808 13:35:27.194113 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.198643 (+ 4530us) service_pool.cc:227] Handling call
0808 13:35:29.660234 (+2461591us) ts_tablet_manager.cc:1145] Deleting tablet a541ec35b1a44109866fabc953f14cc6
0808 13:35:29.660285 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:29.982285 (+322000us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:30.303745 (+321460us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:30.374825 (+ 71080us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2371126}
I20260808 13:35:30.376088 6026 tablet_bootstrap.cc:654] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:30.376411 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 244) took 2568 ms. Trace:
I20260808 13:35:30.376436 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet a541ec35b1a44109866fabc953f14cc6 (table default.Table_0ba9f3241f494f148bd809e66279f480 [id=5b1fc9924efa44b8807de40565cdd419]) successfully deleted
I20260808 13:35:30.376704 7014 rpcz_store.cc:276] 0808 13:35:27.808159 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.825049 (+ 16890us) service_pool.cc:227] Handling call
0808 13:35:30.376065 (+2551016us) ts_tablet_manager.cc:1145] Deleting tablet daa59a18ae624bc8975bee2c460895a7
0808 13:35:30.376103 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:30.376362 (+ 259us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2549854}
W20260808 13:35:30.377825 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet daa59a18ae624bc8975bee2c460895a7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet daa59a18ae624bc8975bee2c460895a7 already in progress: creating tablet
I20260808 13:35:30.377250 5772 leader_election.cc:302] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:30.379087 7240 raft_consensus.cc:2781] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:30.379441 7240 raft_consensus.cc:482] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:30.379737 7240 raft_consensus.cc:3037] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:30.383080 5634 raft_consensus.cc:3037] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:30.387660 7378 log.cc:924] Time spent T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.055s user 0.002s sys 0.000s
I20260808 13:35:30.392030 5634 raft_consensus.cc:2454] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:30.392282 7240 raft_consensus.cc:504] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.400041 5285 leader_election.cc:302] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.400822 7282 tablet.cc:1722] T 344828b16cfc47d5b419f8b569055fa4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:30.402298 7341 raft_consensus.cc:2781] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:30.402823 7341 raft_consensus.cc:686] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:30.402925 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234591e698f8472e90e1f610da37ff3a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:30.403728 7341 consensus_queue.cc:237] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:30.405443 5771 leader_election.cc:341] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.412978 6026 tablet_bootstrap.cc:492] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:30.413410 6026 ts_tablet_manager.cc:1397] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.125s user 0.010s sys 0.000s
W20260808 13:35:30.414366 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 374275us
I20260808 13:35:30.414870 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 250) took 2565 ms. Trace:
I20260808 13:35:30.415544 7014 rpcz_store.cc:276] 0808 13:35:27.811827 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.826477 (+ 14650us) service_pool.cc:227] Handling call
0808 13:35:30.377479 (+2551002us) ts_tablet_manager.cc:1145] Deleting tablet 896389941179456d9d0ed3cd14726f11
0808 13:35:30.377515 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:30.377659 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":146815,"tablet-delete.queue_time_us":2550092}
W20260808 13:35:30.416438 5383 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s user 0.004s sys 0.013s
W20260808 13:35:30.416989 5383 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s user 0.004s sys 0.013s
I20260808 13:35:30.415678 6026 raft_consensus.cc:348] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.425819 6026 raft_consensus.cc:374] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:30.426131 6026 raft_consensus.cc:729] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:30.426791 6026 consensus_queue.cc:260] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.429044 6026 ts_tablet_manager.cc:1428] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
W20260808 13:35:30.430027 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 896389941179456d9d0ed3cd14726f11 because the tablet was not found. No further retry: Not found: Tablet not found: 896389941179456d9d0ed3cd14726f11
I20260808 13:35:30.430495 6026 tablet_bootstrap.cc:492] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:30.431284 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 245) took 2622 ms. Trace:
I20260808 13:35:30.431632 7014 rpcz_store.cc:276] 0808 13:35:27.808434 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.826787 (+ 18353us) service_pool.cc:227] Handling call
0808 13:35:30.431020 (+2604233us) ts_tablet_manager.cc:1145] Deleting tablet 06c59eea10ec45af8e3a0992f9cc74bf
0808 13:35:30.431064 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:30.431228 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":146440,"tablet-delete.queue_time_us":2603153}
W20260808 13:35:30.432911 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 06c59eea10ec45af8e3a0992f9cc74bf because the tablet was not found. No further retry: Not found: Tablet not found: 06c59eea10ec45af8e3a0992f9cc74bf
W20260808 13:35:30.415983 5550 leader_election.cc:341] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.443560 5550 leader_election.cc:302] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.449729 7240 leader_election.cc:288] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:30.468194 6046 tablet_bootstrap.cc:492] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:30.477320 7334 tablet.cc:1722] T 953a2a65dea7468c8356931cabb6bfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:30.488289 7013 ts_tablet_manager.cc:1918] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:30.495486 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 9ea8f5322a404837b91e1d96749926b7
I20260808 13:35:30.496042 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_248e910ea5a34c55978753b74fe71832"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:30.501024 4832 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f66b38e83dab4bf1a3d2d1e088a96988 consensus state has no leader
I20260808 13:35:30.501436 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234591e698f8472e90e1f610da37ff3a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:30.502148 5634 raft_consensus.cc:3037] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:30.511292 5634 raft_consensus.cc:2454] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:30.514658 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea8f5322a404837b91e1d96749926b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:30 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:30.516896 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 248) took 2705 ms. Trace:
W20260808 13:35:30.517127 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_248e910ea5a34c55978753b74fe71832 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:30.517216 7014 rpcz_store.cc:276] 0808 13:35:27.811237 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.931613 (+120376us) service_pool.cc:227] Handling call
0808 13:35:30.465080 (+2533467us) ts_tablet_manager.cc:1145] Deleting tablet b314734698ed423ba9cf4134db5846ec
0808 13:35:30.465139 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:30.516840 (+ 51701us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":37267,"tablet-delete.queue_time_us":2527835}
W20260808 13:35:30.518501 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet b314734698ed423ba9cf4134db5846ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b314734698ed423ba9cf4134db5846ec already in progress: creating tablet
I20260808 13:35:30.519076 6901 raft_consensus.cc:2726] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:30.519845 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 103878us
I20260808 13:35:30.520533 5550 leader_election.cc:302] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.521275 6901 raft_consensus.cc:2726] T 261e45848f424a349b0c5290954c5bc6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:30.522017 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea8f5322a404837b91e1d96749926b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:30 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:30.523608 5550 leader_election.cc:341] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.524003 5550 leader_election.cc:302] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.524672 6901 raft_consensus.cc:2726] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:30.534900 4832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d3498dfbc8c04e42abefc8e337816f9b" } new_table_name: "default.Table_db41a83ae3f34453af64e73302451c19" modify_external_catalogs: true
I20260808 13:35:30.539753 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 255) took 2635 ms. Trace:
I20260808 13:35:30.540150 7014 rpcz_store.cc:276] 0808 13:35:27.904509 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.904861 (+ 352us) service_pool.cc:227] Handling call
0808 13:35:30.518310 (+2613449us) ts_tablet_manager.cc:1145] Deleting tablet 3424ca753bcc4f448d314c56aba3323e
0808 13:35:30.518352 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:30.539688 (+ 21336us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":27856,"tablet-delete.queue_time_us":2571563}
W20260808 13:35:30.541543 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 3424ca753bcc4f448d314c56aba3323e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3424ca753bcc4f448d314c56aba3323e already in progress: creating tablet
I20260808 13:35:30.551548 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_4159935d23b8404ea7b50d91562d2fc7" } modify_external_catalogs: true
I20260808 13:35:30.552091 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_4159935d23b8404ea7b50d91562d2fc7" } modify_external_catalogs: true
W20260808 13:35:30.557156 4832 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 96bed5e6d1aa49a9b14ba86403c985ce consensus state has no leader
I20260808 13:35:30.570739 7014 tablet_replica.cc:330] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:30.571566 7014 raft_consensus.cc:2229] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:30.573319 7014 raft_consensus.cc:2258] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:30.578033 7014 ts_tablet_manager.cc:1905] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:30.583374 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_1c70a97bd7794b7780b8575f70ac6147"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:30.584919 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c70a97bd7794b7780b8575f70ac6147 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:30.590139 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_3961834935a84f0a9646f77db54765ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:30.591665 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3961834935a84f0a9646f77db54765ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:30.596905 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 151940us
I20260808 13:35:30.598248 5772 leader_election.cc:302] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:30.602416 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_7e9aac444b5a43f29dc7608e6ef379f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:30.603946 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e9aac444b5a43f29dc7608e6ef379f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:30.605357 7387 raft_consensus.cc:2781] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:30.605968 7387 raft_consensus.cc:686] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:30.607183 7387 consensus_queue.cc:237] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.576009 6026 tablet_bootstrap.cc:654] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:30.572223 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 48632.9us
I20260808 13:35:30.618382 5550 leader_election.cc:302] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.619241 6901 raft_consensus.cc:2726] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:30.620424 5550 leader_election.cc:341] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.620880 5550 leader_election.cc:302] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.621644 6901 raft_consensus.cc:2726] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:30.642112 6033 tablet_bootstrap.cc:492] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:30.642546 6033 ts_tablet_manager.cc:1397] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.333s user 0.025s sys 0.050s
I20260808 13:35:30.654934 6033 raft_consensus.cc:348] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.655649 6033 raft_consensus.cc:374] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:30.655931 6033 raft_consensus.cc:729] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:30.656481 6033 consensus_queue.cc:260] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.658296 6033 ts_tablet_manager.cc:1428] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260808 13:35:30.660411 7375 tablet.cc:1722] T 953a2a65dea7468c8356931cabb6bfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:30.661892 6901 raft_consensus.cc:482] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:30.644932 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 128506us
I20260808 13:35:30.651553 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea8f5322a404837b91e1d96749926b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:30 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:30.662364 6901 raft_consensus.cc:504] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.663475 7013 log.cc:1196] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/a541ec35b1a44109866fabc953f14cc6
I20260808 13:35:30.664352 7013 ts_tablet_manager.cc:1939] T a541ec35b1a44109866fabc953f14cc6 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:30.664476 6901 leader_election.cc:288] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:30.665349 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 255) took 2936 ms. Trace:
I20260808 13:35:30.665706 7013 rpcz_store.cc:276] 0808 13:35:27.729365 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.761248 (+ 31883us) service_pool.cc:227] Handling call
0808 13:35:29.601305 (+1840057us) ts_tablet_manager.cc:1145] Deleting tablet a541ec35b1a44109866fabc953f14cc6
0808 13:35:29.601345 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:30.389215 (+787870us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:30.487983 (+ 98768us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:30.665297 (+177314us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":233290,"tablet-delete.queue_time_us":1827612}
I20260808 13:35:30.667276 7013 tablet_replica.cc:330] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:30.671815 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69642fdba1f145438b04eae9f8154101" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:30.684363 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet a541ec35b1a44109866fabc953f14cc6 (table default.Table_0ba9f3241f494f148bd809e66279f480 [id=5b1fc9924efa44b8807de40565cdd419]) successfully deleted
I20260808 13:35:30.705107 6901 raft_consensus.cc:482] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:30.705976 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6e898d8b2bda40a8b235d59bc101c5e5" } new_table_name: "default.Table_b8b2109694134a6bbc7c2bc43b158e1c" modify_external_catalogs: true
I20260808 13:35:30.705754 6901 raft_consensus.cc:504] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.708065 6901 leader_election.cc:288] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:30.708647 6033 tablet_bootstrap.cc:492] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:30.719498 4838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_ca3570782c0f48a88952f5f442fdbde6" } modify_external_catalogs: true
I20260808 13:35:30.720064 4838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_ca3570782c0f48a88952f5f442fdbde6" } modify_external_catalogs: true
I20260808 13:35:30.728010 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ba43d1479a4f9e9d08bdfa29e7157b" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:30.729782 5550 leader_election.cc:341] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:30.730523 5550 leader_election.cc:341] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.743700 6033 tablet_bootstrap.cc:654] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:30.764243 4832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_13bc8e545bba490b8fef6864a18684a0" } new_table_name: "default.Table_953730643273433c8353604101a4d237" modify_external_catalogs: true
I20260808 13:35:30.769165 6033 tablet_bootstrap.cc:492] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:30.769680 6033 ts_tablet_manager.cc:1397] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.061s user 0.011s sys 0.000s
I20260808 13:35:30.772913 6033 raft_consensus.cc:348] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:30.773787 6033 raft_consensus.cc:374] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:30.774098 6033 raft_consensus.cc:729] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:30.775224 7014 ts_tablet_manager.cc:1918] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:30.775657 7014 log.cc:1196] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/6826e4e4db2043b3a4fe9e7a9359cab6
I20260808 13:35:30.776422 7014 ts_tablet_manager.cc:1939] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
W20260808 13:35:30.776872 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5663fa6a1bb148c3afc9340477b05a4b consensus state has no leader
I20260808 13:35:30.782354 6033 consensus_queue.cc:260] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:30.783917 6026 tablet_bootstrap.cc:492] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:30.784346 6026 ts_tablet_manager.cc:1397] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.354s user 0.013s sys 0.000s
I20260808 13:35:30.788771 6026 raft_consensus.cc:348] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.789446 6026 raft_consensus.cc:374] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:30.789685 6026 raft_consensus.cc:729] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:30.790516 6026 consensus_queue.cc:260] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.792332 6033 ts_tablet_manager.cc:1428] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.022s user 0.009s sys 0.000s
I20260808 13:35:30.797927 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_025eb6ef4e024bec917c3272ba1b8c35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:30.799726 6033 tablet_bootstrap.cc:492] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:30.799731 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_025eb6ef4e024bec917c3272ba1b8c35 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:30.815385 6026 ts_tablet_manager.cc:1428] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.031s user 0.008s sys 0.000s
I20260808 13:35:30.819177 6033 tablet_bootstrap.cc:654] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:30.677151 6046 tablet_bootstrap.cc:654] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:30.678360 7309 raft_consensus.cc:482] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 98dd96f425fc4ab3a6a54f2dfaaf9fab)
I20260808 13:35:30.691073 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69642fdba1f145438b04eae9f8154101" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:30.710141 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ba43d1479a4f9e9d08bdfa29e7157b" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:30.832798 7199 ts_tablet_manager.cc:1918] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:30.833444 7199 log.cc:1196] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/d3682c9c656c4556bc13ae3aeddd7c5e
I20260808 13:35:30.834381 7199 ts_tablet_manager.cc:1939] T d3682c9c656c4556bc13ae3aeddd7c5e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:30.820147 7013 raft_consensus.cc:2229] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:30.835039 7013 raft_consensus.cc:2258] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:30.838970 7013 ts_tablet_manager.cc:1905] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:30.829365 7309 raft_consensus.cc:504] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.843425 7309 leader_election.cc:288] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:30.845368 5285 leader_election.cc:341] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 024e7f2daaad4f48a79b94344ff35e98
W20260808 13:35:30.847285 5283 leader_election.cc:341] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: 024e7f2daaad4f48a79b94344ff35e98
I20260808 13:35:30.847630 5283 leader_election.cc:302] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.842051 7399 tablet.cc:1722] T 344828b16cfc47d5b419f8b569055fa4 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260808 13:35:30.852640 5551 leader_election.cc:341] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.853044 5551 leader_election.cc:302] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.854187 6901 raft_consensus.cc:2726] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:30.858709 5551 leader_election.cc:341] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.859092 5551 leader_election.cc:302] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:30.860051 6901 raft_consensus.cc:2726] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:30.866339 6033 tablet_bootstrap.cc:492] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:30.866928 6033 ts_tablet_manager.cc:1397] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.067s user 0.016s sys 0.010s
I20260808 13:35:30.870549 6033 raft_consensus.cc:348] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:30.871260 6033 raft_consensus.cc:374] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:30.871507 6033 raft_consensus.cc:729] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:30.872324 6033 consensus_queue.cc:260] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:30.861406 7309 raft_consensus.cc:2726] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:30.876782 6033 ts_tablet_manager.cc:1428] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260808 13:35:30.882799 7391 raft_consensus.cc:482] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:30.883287 7391 raft_consensus.cc:504] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.885516 7391 leader_election.cc:288] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:30.888577 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd6945ecccc4684a3fa5b2a1f07057f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:30.889176 5637 raft_consensus.cc:2454] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:30.890394 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd6945ecccc4684a3fa5b2a1f07057f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:30.891090 5772 leader_election.cc:302] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:35:30.892169 5771 leader_election.cc:341] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.893122 6026 tablet_bootstrap.cc:492] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:30.893496 7391 raft_consensus.cc:2781] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:30.893945 7391 raft_consensus.cc:482] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:30.893769 7398 tablet.cc:1722] T 953a2a65dea7468c8356931cabb6bfc9 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:30.894316 7391 raft_consensus.cc:3037] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:30.778509 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 246) took 2969 ms. Trace:
I20260808 13:35:30.903519 7014 rpcz_store.cc:276] 0808 13:35:27.808730 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.829710 (+ 20980us) service_pool.cc:227] Handling call
0808 13:35:30.542510 (+2712800us) ts_tablet_manager.cc:1145] Deleting tablet 6826e4e4db2043b3a4fe9e7a9359cab6
0808 13:35:30.542552 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:30.739575 (+197023us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:30.774954 (+ 35379us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:30.778445 (+ 3491us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28059,"tablet-delete.queue_time_us":2595647}
I20260808 13:35:30.904232 6033 tablet_bootstrap.cc:492] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:30.906776 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 6826e4e4db2043b3a4fe9e7a9359cab6 (table default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]) successfully deleted
I20260808 13:35:30.907859 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 249) took 3096 ms. Trace:
I20260808 13:35:30.910694 7014 rpcz_store.cc:276] 0808 13:35:27.811533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.975571 (+164038us) service_pool.cc:227] Handling call
0808 13:35:30.907561 (+2931990us) ts_tablet_manager.cc:1145] Deleting tablet 9260370cf8cb46a7b699581e4e8d46db
0808 13:35:30.907605 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:30.907803 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2891269}
W20260808 13:35:30.912014 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 9260370cf8cb46a7b699581e4e8d46db because the tablet was not found. No further retry: Not found: Tablet not found: 9260370cf8cb46a7b699581e4e8d46db
I20260808 13:35:30.917423 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 258) took 2885 ms. Trace:
I20260808 13:35:30.918179 7014 tablet_replica.cc:330] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:30.921022 7014 raft_consensus.cc:2229] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:30.926920 7199 rpcz_store.cc:276] 0808 13:35:28.032083 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.032256 (+ 173us) service_pool.cc:227] Handling call
0808 13:35:29.754798 (+1722542us) ts_tablet_manager.cc:1145] Deleting tablet d3682c9c656c4556bc13ae3aeddd7c5e
0808 13:35:29.754877 (+ 79us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:30.405186 (+650309us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:30.832485 (+427299us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:30.917363 (+ 84878us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1720158}
I20260808 13:35:30.928033 6033 tablet_bootstrap.cc:654] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:30.928356 7199 tablet_replica.cc:330] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:30.929177 7199 raft_consensus.cc:2229] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:30.930117 7199 raft_consensus.cc:2258] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:30.931806 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_83babd1cd9304044b4bdb5cf0ee909a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:30.933108 7391 raft_consensus.cc:504] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.935870 7014 raft_consensus.cc:2258] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:30.936771 6046 tablet_bootstrap.cc:492] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:30.937232 6046 ts_tablet_manager.cc:1397] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.469s user 0.012s sys 0.006s
I20260808 13:35:30.941421 7387 raft_consensus.cc:482] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:30.942027 7387 raft_consensus.cc:504] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.945143 7387 leader_election.cc:288] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:30.947984 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127e8d0323954ef788018bcb91582951" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:30.948663 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_1ab5d21bca4a47978a8f5483f2ba10b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:30.949014 5383 raft_consensus.cc:2454] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
W20260808 13:35:30.950148 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ab5d21bca4a47978a8f5483f2ba10b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:30.950665 5771 leader_election.cc:302] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:30.951498 7241 raft_consensus.cc:482] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:30.952185 7241 raft_consensus.cc:504] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:30.956015 6046 raft_consensus.cc:348] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.956692 6046 raft_consensus.cc:374] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:30.956506 7241 leader_election.cc:288] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:30.956940 6046 raft_consensus.cc:729] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:30.956924 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_e916116a3ec24469b4670c87eeaac98e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:30.957252 7241 raft_consensus.cc:2781] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:30.959640 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8514ca242bb2418d8ddbb88548b21832" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:30.961177 5771 leader_election.cc:341] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.958770 7241 raft_consensus.cc:482] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:30.962654 7241 raft_consensus.cc:3037] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:30.963443 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8514ca242bb2418d8ddbb88548b21832" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:30.939572 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd6945ecccc4684a3fa5b2a1f07057f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:30.967867 5771 leader_election.cc:341] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.970479 7391 leader_election.cc:288] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:30.971208 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127e8d0323954ef788018bcb91582951" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:30.971580 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83babd1cd9304044b4bdb5cf0ee909a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:30.971949 5637 raft_consensus.cc:2454] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
W20260808 13:35:30.973135 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e916116a3ec24469b4670c87eeaac98e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:30.974360 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd6945ecccc4684a3fa5b2a1f07057f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:30.975049 5637 raft_consensus.cc:3037] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:30.976127 7241 raft_consensus.cc:504] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:30.979301 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127e8d0323954ef788018bcb91582951" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:30.979866 5378 raft_consensus.cc:3037] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:30.980633 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_912bf698cf184d5690eeba3080b2b9bf" } new_table_name: "default.Table_86d171579f64458497d484e10e5f630a" modify_external_catalogs: true
W20260808 13:35:30.983096 5772 leader_election.cc:341] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:30.983608 5772 leader_election.cc:302] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:30.984611 6033 tablet_bootstrap.cc:492] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:30.985085 6033 ts_tablet_manager.cc:1397] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.081s user 0.009s sys 0.004s
I20260808 13:35:30.985175 7014 ts_tablet_manager.cc:1905] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:30.987150 7391 raft_consensus.cc:2726] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:30.990114 6033 raft_consensus.cc:348] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:30.992517 6033 raft_consensus.cc:374] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:30.993009 6033 raft_consensus.cc:729] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:30.995589 6033 consensus_queue.cc:260] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:30.997601 6046 consensus_queue.cc:260] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.000968 6033 ts_tablet_manager.cc:1428] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
W20260808 13:35:31.003471 4837 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fcf907a9f19f4d64b27be946011a8ddb consensus state has no leader
I20260808 13:35:31.004918 7199 ts_tablet_manager.cc:1905] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:31.005070 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a73e98a8b20144508a5a8d451f4bb302 consensus state has no leader
I20260808 13:35:31.001732 7241 leader_election.cc:288] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:31.024039 6046 ts_tablet_manager.cc:1428] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.087s user 0.008s sys 0.000s
I20260808 13:35:31.025041 6046 tablet_bootstrap.cc:492] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:31.035830 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 52765.1us
I20260808 13:35:31.038332 6033 tablet_bootstrap.cc:492] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:31.042008 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127e8d0323954ef788018bcb91582951" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:31.042760 5640 raft_consensus.cc:3037] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:31.043032 7013 ts_tablet_manager.cc:1918] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:31.047859 7241 consensus_queue.cc:1035] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:31.053009 6033 tablet_bootstrap.cc:654] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:31.054186 7013 log.cc:1196] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/06c59eea10ec45af8e3a0992f9cc74bf
I20260808 13:35:31.059037 5378 raft_consensus.cc:2454] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:31.060316 5771 leader_election.cc:302] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:31.061055 7241 raft_consensus.cc:2781] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:31.061460 7241 raft_consensus.cc:686] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:31.062234 7241 consensus_queue.cc:237] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.058458 7013 ts_tablet_manager.cc:1939] T 06c59eea10ec45af8e3a0992f9cc74bf P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:31.074572 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 256) took 3312 ms. Trace:
I20260808 13:35:31.093147 7309 consensus_queue.cc:1035] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:31.098318 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_1e7d0ae2ebb1437c83136cc4e91e8a23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:31.112136 6046 tablet_bootstrap.cc:654] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:31.074980 7013 rpcz_store.cc:276] 0808 13:35:27.762180 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.762339 (+ 159us) service_pool.cc:227] Handling call
0808 13:35:30.667140 (+2904801us) ts_tablet_manager.cc:1145] Deleting tablet 06c59eea10ec45af8e3a0992f9cc74bf
0808 13:35:30.667177 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:30.941993 (+274816us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.042807 (+100814us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.074508 (+ 31701us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2893180}
I20260808 13:35:31.114413 7013 tablet_replica.cc:330] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:31.115111 7013 raft_consensus.cc:2229] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:31.115564 7013 raft_consensus.cc:2258] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:31.115582 7241 raft_consensus.cc:482] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:31.116183 7241 raft_consensus.cc:504] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.117228 7013 ts_tablet_manager.cc:1905] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:31.117482 6026 tablet_bootstrap.cc:654] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:31.119529 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6826e4e4db2043b3a4fe9e7a9359cab6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:31.123701 5640 raft_consensus.cc:2454] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:31.126896 7391 raft_consensus.cc:482] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:31.127489 7391 raft_consensus.cc:504] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.131402 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9bc338093f403881d329a7e913c9f7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:31.130218 7391 leader_election.cc:288] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:31.132076 5636 raft_consensus.cc:2454] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:31.133559 5772 leader_election.cc:302] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:31.133834 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9bc338093f403881d329a7e913c9f7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:31.134414 7391 raft_consensus.cc:2781] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:31.134945 7391 raft_consensus.cc:482] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:31.135412 7391 raft_consensus.cc:3037] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:31.135601 5771 leader_election.cc:341] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:31.139484 4841 master_service.cc:946] ReplaceTablet: received request to replace tablet fc9b2e69fe27431dac9eb5ecb9e3e23e from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:31.145239 7241 leader_election.cc:288] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:31.145081 7391 raft_consensus.cc:504] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:31.150347 5772 leader_election.cc:341] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 6826e4e4db2043b3a4fe9e7a9359cab6
I20260808 13:35:31.150774 5772 leader_election.cc:302] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:31.152963 6033 tablet_bootstrap.cc:492] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:31.153369 6033 ts_tablet_manager.cc:1397] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.115s user 0.005s sys 0.005s
I20260808 13:35:31.153201 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9bc338093f403881d329a7e913c9f7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:31.155596 5771 leader_election.cc:341] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:31.157505 7391 leader_election.cc:288] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:31.162911 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e7d0ae2ebb1437c83136cc4e91e8a23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:31.113847 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 06c59eea10ec45af8e3a0992f9cc74bf (table default.Table_0c2610996a984e98b9d1899f23a40bf5 [id=d158bea8f5fc445e98a01e20b8a53651]) successfully deleted
W20260808 13:35:31.165360 4820 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 51534us
I20260808 13:35:31.169639 7241 raft_consensus.cc:2726] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260808 13:35:31.215472 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 65149.2us
I20260808 13:35:31.217222 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9bc338093f403881d329a7e913c9f7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:31.217913 5640 raft_consensus.cc:3037] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:31.228811 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 1f0a6d9bd2c74d0790badd7977f027e3 (DEFAULT_TABLE table=default.Table_ff9135dbb8a84d158dc9f2f209d8d741 [id=6416856581b645649336cd4051e0e774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.229398 7014 ts_tablet_manager.cc:1918] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:31.229797 7014 log.cc:1196] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/4625b632d67244fc86e48d739dde707e
I20260808 13:35:31.231240 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1c70a97bd7794b7780b8575f70ac6147" } new_table_name: "default.Table_9758d29f1a0b4ea5bceef3d5ce652ca1" modify_external_catalogs: true
I20260808 13:35:31.233682 6046 tablet_bootstrap.cc:492] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:31.233770 5640 raft_consensus.cc:2454] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:31.234087 6046 ts_tablet_manager.cc:1397] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.209s user 0.009s sys 0.000s
I20260808 13:35:31.235131 5637 raft_consensus.cc:2454] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:31.240953 7387 raft_consensus.cc:482] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:31.241420 7387 raft_consensus.cc:504] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.243317 7387 leader_election.cc:288] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:31.244834 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9423b380a62c467895e7fdff14756586" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:31.246279 5771 leader_election.cc:341] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:31.247165 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 1f0a6d9bd2c74d0790badd7977f027e3 (DEFAULT_TABLE table=default.Table_ff9135dbb8a84d158dc9f2f209d8d741 [id=6416856581b645649336cd4051e0e774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.248346 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f0a6d9bd2c74d0790badd7977f027e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.236894 6046 raft_consensus.cc:348] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.250478 6046 raft_consensus.cc:374] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:31.250777 6046 raft_consensus.cc:729] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:31.254912 6046 consensus_queue.cc:260] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.261971 5834 tablet_service.cc:1467] Processing CreateTablet for tablet ffbdf03ebe4e48a98450e90a3a78181c (DEFAULT_TABLE table=default.Table_41a67a20ea754c6fb307fb434f9711ae [id=212e1314b3c04c4d8dd06d5730fa25cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.263880 6046 ts_tablet_manager.cc:1428] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260808 13:35:31.265512 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f0a6d9bd2c74d0790badd7977f027e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.265460 6033 raft_consensus.cc:348] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.266284 6033 raft_consensus.cc:374] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:31.266523 6033 raft_consensus.cc:729] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:31.267287 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 553a01330cfe4dd38a61860dc943fbbc (DEFAULT_TABLE table=default.Table_a8ff8993913d4a05a3d8ab3bb339b7ac [id=8ac0944cbe564a949c732ec146a9cf94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.271397 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 553a01330cfe4dd38a61860dc943fbbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.272228 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_ddfcf011ede6442fbc7a45c799fc2cc6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:31.272411 7309 consensus_queue.cc:1035] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:31.273878 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddfcf011ede6442fbc7a45c799fc2cc6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:31.274906 5772 leader_election.cc:302] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:31.275348 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffbdf03ebe4e48a98450e90a3a78181c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.275636 7241 raft_consensus.cc:2781] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:31.277314 7013 ts_tablet_manager.cc:1918] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:31.277686 7013 log.cc:1196] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/024e7f2daaad4f48a79b94344ff35e98
I20260808 13:35:31.276477 7241 raft_consensus.cc:686] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:31.279394 7241 consensus_queue.cc:237] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.280107 7013 ts_tablet_manager.cc:1939] T 024e7f2daaad4f48a79b94344ff35e98 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:31.280856 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:31.285588 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 41347e0da1654dde8b48c896db6a0915 (DEFAULT_TABLE table=default.Table_280edf174ae84743aac082cfd2788315 [id=ae895d0078c748d78940cf2152e0e635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.286712 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41347e0da1654dde8b48c896db6a0915. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.295576 6046 tablet_bootstrap.cc:492] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:31.298834 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 257) took 3536 ms. Trace:
I20260808 13:35:31.299144 7013 rpcz_store.cc:276] 0808 13:35:27.762563 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.789820 (+ 27257us) service_pool.cc:227] Handling call
0808 13:35:31.114265 (+3324445us) ts_tablet_manager.cc:1145] Deleting tablet 024e7f2daaad4f48a79b94344ff35e98
0808 13:35:31.114309 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:31.183764 (+ 69455us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.218593 (+ 34829us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.298774 (+ 80181us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3323829}
I20260808 13:35:31.300627 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 263) took 3314 ms. Trace:
I20260808 13:35:31.301221 6046 tablet_bootstrap.cc:654] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:31.303249 7387 consensus_queue.cc:1035] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:31.306236 5772 leader_election.cc:302] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:31.307180 7387 raft_consensus.cc:2781] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:31.307663 7387 raft_consensus.cc:686] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:31.308101 5336 tablet_service.cc:1467] Processing CreateTablet for tablet 1f0a6d9bd2c74d0790badd7977f027e3 (DEFAULT_TABLE table=default.Table_ff9135dbb8a84d158dc9f2f209d8d741 [id=6416856581b645649336cd4051e0e774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.308511 7387 consensus_queue.cc:237] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.309355 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f0a6d9bd2c74d0790badd7977f027e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.230561 7014 ts_tablet_manager.cc:1939] T 4625b632d67244fc86e48d739dde707e P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:31.267254 6033 consensus_queue.cc:260] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.328229 5334 tablet_service.cc:1467] Processing CreateTablet for tablet ffbdf03ebe4e48a98450e90a3a78181c (DEFAULT_TABLE table=default.Table_41a67a20ea754c6fb307fb434f9711ae [id=212e1314b3c04c4d8dd06d5730fa25cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.329437 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffbdf03ebe4e48a98450e90a3a78181c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.344090 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 553a01330cfe4dd38a61860dc943fbbc (DEFAULT_TABLE table=default.Table_a8ff8993913d4a05a3d8ab3bb339b7ac [id=8ac0944cbe564a949c732ec146a9cf94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.300915 7013 rpcz_store.cc:276] 0808 13:35:27.986775 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:27.986991 (+ 216us) service_pool.cc:227] Handling call
0808 13:35:31.300286 (+3313295us) ts_tablet_manager.cc:1145] Deleting tablet 3424ca753bcc4f448d314c56aba3323e
0808 13:35:31.300325 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:31.300575 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3312592}
I20260808 13:35:31.345155 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 553a01330cfe4dd38a61860dc943fbbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.347518 5345 tablet_service.cc:1467] Processing CreateTablet for tablet 41347e0da1654dde8b48c896db6a0915 (DEFAULT_TABLE table=default.Table_280edf174ae84743aac082cfd2788315 [id=ae895d0078c748d78940cf2152e0e635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.210371 7199 ts_tablet_manager.cc:1918] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:31.348160 7199 log.cc:1196] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/e992a0294c404cfb9541b57782a05fc9
I20260808 13:35:31.348454 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41347e0da1654dde8b48c896db6a0915. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.349252 7199 ts_tablet_manager.cc:1939] T e992a0294c404cfb9541b57782a05fc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:31.354882 6033 ts_tablet_manager.cc:1428] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.201s user 0.008s sys 0.000s
I20260808 13:35:31.357955 5845 tablet_service.cc:1467] Processing CreateTablet for tablet a11a2666f0b44d70a08a9195e8fcf69d (DEFAULT_TABLE table=default.Table_6eb3cce7943e47dc9749d45cfae01235 [id=7822348d6a1d42d19db3da14f02c2302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.312974 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_4ef3501cc89b4779b73a48142fd01d36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:31.359318 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a11a2666f0b44d70a08a9195e8fcf69d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:31.359702 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ef3501cc89b4779b73a48142fd01d36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:31.360311 7013 tablet_replica.cc:330] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:31.350890 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 3272708a835c432c878cc330e44490c5
I20260808 13:35:31.364260 6033 tablet_bootstrap.cc:492] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:31.365240 5832 tablet_service.cc:1514] Processing DeleteTablet for tablet 3272708a835c432c878cc330e44490c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:30 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:31.365849 7013 raft_consensus.cc:2229] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:31.366359 7013 raft_consensus.cc:2258] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:31.368091 7013 ts_tablet_manager.cc:1905] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:31.364853 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_2a111b21bed74a8bb3c9822dff0c2cc6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:31.371189 5283 consensus_peers.cc:487] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:20.955240 6848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:35:20.955698 6859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:35:31.371827 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a111b21bed74a8bb3c9822dff0c2cc6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:31.371721 7309 consensus_queue.cc:1035] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:20.942478 6450 server_base.cc:1034] running on GCE node
I20260808 13:35:31.383075 5603 tablet_service.cc:1467] Processing CreateTablet for tablet ffbdf03ebe4e48a98450e90a3a78181c (DEFAULT_TABLE table=default.Table_41a67a20ea754c6fb307fb434f9711ae [id=212e1314b3c04c4d8dd06d5730fa25cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.383338 5615 tablet_service.cc:1467] Processing CreateTablet for tablet 553a01330cfe4dd38a61860dc943fbbc (DEFAULT_TABLE table=default.Table_a8ff8993913d4a05a3d8ab3bb339b7ac [id=8ac0944cbe564a949c732ec146a9cf94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.384187 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffbdf03ebe4e48a98450e90a3a78181c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.386854 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 3272708a835c432c878cc330e44490c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:30 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:31.388463 6450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:35:31.395033 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9423b380a62c467895e7fdff14756586" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:31.395421 5607 tablet_service.cc:1467] Processing CreateTablet for tablet 41347e0da1654dde8b48c896db6a0915 (DEFAULT_TABLE table=default.Table_280edf174ae84743aac082cfd2788315 [id=ae895d0078c748d78940cf2152e0e635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.395625 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_e101c061f2bf4821b6fe2c6108432735 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]). Expected version 3 got 2
I20260808 13:35:31.396762 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_9d2867cdebf84440b7aa21a71faba9c4 [id=0ddf25cf69f449cf8c7516cb84736b3d]). Expected version 2 got 1
I20260808 13:35:31.397192 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 553a01330cfe4dd38a61860dc943fbbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.398689 5608 tablet_service.cc:1467] Processing CreateTablet for tablet a11a2666f0b44d70a08a9195e8fcf69d (DEFAULT_TABLE table=default.Table_6eb3cce7943e47dc9749d45cfae01235 [id=7822348d6a1d42d19db3da14f02c2302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.399796 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a11a2666f0b44d70a08a9195e8fcf69d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:31.402884 4820 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 55654.7us
I20260808 13:35:31.403508 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 024e7f2daaad4f48a79b94344ff35e98 (table default.Table_fad1a84e78af4cfe8c7ceee0e14a75e5 [id=cda1418c2bf5405abda61222d65589cf]) successfully deleted
I20260808 13:35:31.404193 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41347e0da1654dde8b48c896db6a0915. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:31.404168 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 3424ca753bcc4f448d314c56aba3323e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3424ca753bcc4f448d314c56aba3323e already in progress: creating tablet
I20260808 13:35:31.412611 5383 raft_consensus.cc:3037] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:31.398550 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:31.423508 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 259) took 3386 ms. Trace:
I20260808 13:35:31.423874 7199 rpcz_store.cc:276] 0808 13:35:28.036891 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.037041 (+ 150us) service_pool.cc:227] Handling call
0808 13:35:30.928172 (+2891131us) ts_tablet_manager.cc:1145] Deleting tablet e992a0294c404cfb9541b57782a05fc9
0808 13:35:30.928224 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:31.153026 (+224802us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.210124 (+ 57098us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.423444 (+213320us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2843403}
I20260808 13:35:31.425086 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet e992a0294c404cfb9541b57782a05fc9 (table default.Table_6f4c6712b74748f58390a05ec941a26a [id=2b4fc1c958044df7939df70bc136d956]) successfully deleted
I20260808 13:35:31.431842 6033 tablet_bootstrap.cc:654] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:31.433512 7387 consensus_queue.cc:1035] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:31.433437 5842 tablet_service.cc:1467] Processing CreateTablet for tablet a9ea181156f64388ade8a07708b2e814 (DEFAULT_TABLE table=default.Table_577d84019e77413d9ff7373a8315d384 [id=6f200cd5eaff4a63a8b2672962db8c78]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:31.449364 5772 leader_election.cc:341] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:31.449841 5772 leader_election.cc:302] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:31.451110 7387 raft_consensus.cc:2726] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:31.500347 6033 tablet_bootstrap.cc:492] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:31.500844 6033 ts_tablet_manager.cc:1397] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.145s user 0.007s sys 0.003s
I20260808 13:35:31.505625 6026 tablet_bootstrap.cc:492] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:31.506057 6026 ts_tablet_manager.cc:1397] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.613s user 0.010s sys 0.020s
I20260808 13:35:31.508297 6026 raft_consensus.cc:348] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.508935 6026 raft_consensus.cc:374] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:31.509161 6026 raft_consensus.cc:729] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:31.509682 6026 consensus_queue.cc:260] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.514163 6026 ts_tablet_manager.cc:1428] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260808 13:35:31.515139 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 256) took 3335 ms. Trace:
I20260808 13:35:31.516036 7014 rpcz_store.cc:276] 0808 13:35:28.179829 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.179982 (+ 153us) service_pool.cc:227] Handling call
0808 13:35:30.918032 (+2738050us) ts_tablet_manager.cc:1145] Deleting tablet 4625b632d67244fc86e48d739dde707e
0808 13:35:30.918072 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:31.053087 (+135015us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.229161 (+176074us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.515070 (+285909us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2727455}
I20260808 13:35:31.505739 6033 raft_consensus.cc:348] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.523348 6033 raft_consensus.cc:374] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:31.523608 6033 raft_consensus.cc:729] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:31.523645 6026 tablet_bootstrap.cc:492] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:31.528772 6026 tablet_bootstrap.cc:654] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:31.532495 6033 consensus_queue.cc:260] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.540874 7013 ts_tablet_manager.cc:1918] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:31.540932 7199 tablet_replica.cc:330] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:31.541810 7199 raft_consensus.cc:2229] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:31.542305 7199 raft_consensus.cc:2258] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:31.542568 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9ea181156f64388ade8a07708b2e814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.529389 7014 tablet_replica.cc:330] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:31.543504 7014 raft_consensus.cc:2229] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:31.544031 7014 raft_consensus.cc:2258] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:31.547307 7014 ts_tablet_manager.cc:1905] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:31.549204 7387 raft_consensus.cc:482] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:31.549623 7387 raft_consensus.cc:504] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.551616 7387 leader_election.cc:288] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:31.554224 6033 ts_tablet_manager.cc:1428] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.053s user 0.006s sys 0.000s
I20260808 13:35:31.554186 4832 master_service.cc:946] ReplaceTablet: received request to replace tablet fc9b2e69fe27431dac9eb5ecb9e3e23e from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:31.565603 6026 tablet_bootstrap.cc:492] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:31.566030 6026 ts_tablet_manager.cc:1397] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.001s
I20260808 13:35:31.568311 6026 raft_consensus.cc:348] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.568984 6026 raft_consensus.cc:374] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:31.569231 6026 raft_consensus.cc:729] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:31.569788 6026 consensus_queue.cc:260] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.577483 7013 log.cc:1196] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/4625b632d67244fc86e48d739dde707e
I20260808 13:35:31.578401 7013 ts_tablet_manager.cc:1939] T 4625b632d67244fc86e48d739dde707e P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:31.579460 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 265) took 3351 ms. Trace:
I20260808 13:35:31.583154 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 62d68ab73bae4dd9af46e907b1fd198c
I20260808 13:35:31.584565 4833 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 332) took 1033 ms. Trace:
I20260808 13:35:31.591199 5832 tablet_service.cc:1467] Processing CreateTablet for tablet c2c50fd06b8841c8b5a944401b495d2e (DEFAULT_TABLE table=default.Table_323ff643b74d4f8e8a82e3fe8d1fc10c [id=4cbe915d8abf43369d194d4087999bc5]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:31.593397 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ce39289278cb44cc86d41c3abb516a1d has no consensus state
I20260808 13:35:31.584934 4833 rpcz_store.cc:276] 0808 13:35:30.550961 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.551088 (+ 127us) service_pool.cc:227] Handling call
0808 13:35:30.552611 (+ 1523us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:30.554220 (+ 1609us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:30.554364 (+ 144us) catalog_manager.cc:2761] Locking tablets
0808 13:35:31.221623 (+667259us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:31.582723 (+361100us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:31.582957 (+ 234us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:31.583068 (+ 111us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:31.584505 (+ 1437us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:31.226377 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:31.242001 (+ 15624us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:31.242029 (+ 28us) write_op.cc:435] Acquired schema lock
0808 13:35:31.242042 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:31.242402 (+ 360us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:31.242441 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:31.242463 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:31.253745 (+ 11282us) tablet.cc:699] Row locks acquired
0808 13:35:31.253771 (+ 26us) write_op.cc:260] PREPARE: finished
0808 13:35:31.253901 (+ 130us) write_op.cc:270] Start()
0808 13:35:31.253992 (+ 91us) write_op.cc:276] Timestamp: P: 1786196131253859 usec, L: 0
0808 13:35:31.254009 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:31.254638 (+ 629us) log.cc:842] Serialized 731 byte log entry
0808 13:35:31.579287 (+324649us) op_driver.cc:444] REPLICATION: finished
0808 13:35:31.579980 (+ 693us) write_op.cc:301] APPLY: starting
0808 13:35:31.580038 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:31.580147 (+ 109us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:31.580247 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:31.580498 (+ 251us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:31.580642 (+ 144us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:31.580682 (+ 40us) write_op.cc:312] APPLY: finished
0808 13:35:31.580885 (+ 203us) log.cc:842] Serialized 29 byte log entry
0808 13:35:31.581038 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:31.581105 (+ 67us) write_op.cc:454] Released schema lock
0808 13:35:31.581137 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:31.581312 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:31.581343 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":524,"apply.run_cpu_time_us":1555,"apply.run_wall_time_us":1555,"num_ops":2,"prepare.queue_time_us":2998,"prepare.run_cpu_time_us":2382,"prepare.run_wall_time_us":28486,"replication_time_us":325189}]]}
I20260808 13:35:31.603706 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 62d68ab73bae4dd9af46e907b1fd198c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:30 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:31.605206 5836 tablet_service.cc:1467] Processing CreateTablet for tablet b6b421ff1f094d8689010b4ed6c3f7af (DEFAULT_TABLE table=default.Table_b5830adfb9064ab4b32c29c4c1a03066 [id=84e585111c464794ac7ee0e88227ee7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.581557 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc9b2e69fe27431dac9eb5ecb9e3e23e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:31.596333 5354 tablet_service.cc:1467] Processing CreateTablet for tablet a11a2666f0b44d70a08a9195e8fcf69d (DEFAULT_TABLE table=default.Table_6eb3cce7943e47dc9749d45cfae01235 [id=7822348d6a1d42d19db3da14f02c2302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.613019 5834 tablet_service.cc:1467] Processing CreateTablet for tablet 9dee4b3b14264f3aad0896f629ab5b27 (DEFAULT_TABLE table=default.Table_5ede676dcd5843a3b896699541bf6d01 [id=11f8a63441444ac1a7aa7165cbe4b696]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.613431 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a11a2666f0b44d70a08a9195e8fcf69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.613479 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 9dee4b3b14264f3aad0896f629ab5b27 (DEFAULT_TABLE table=default.Table_5ede676dcd5843a3b896699541bf6d01 [id=11f8a63441444ac1a7aa7165cbe4b696]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.614454 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dee4b3b14264f3aad0896f629ab5b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.593865 4840 catalog_manager.cc:5495] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:31.615573 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 290081bbe4164acd8808041b76441f0d (DEFAULT_TABLE table=default.Table_9936d69a1f7b4853a7c37d5e8da2077f [id=a7e1be7320c34f3bab091c92ec2c1f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.616120 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:31.617142 4840 catalog_manager.cc:5495] T 261e45848f424a349b0c5290954c5bc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:31.597705 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet 3272708a835c432c878cc330e44490c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:30 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:31.598798 5336 tablet_service.cc:1467] Processing CreateTablet for tablet a9ea181156f64388ade8a07708b2e814 (DEFAULT_TABLE table=default.Table_577d84019e77413d9ff7373a8315d384 [id=6f200cd5eaff4a63a8b2672962db8c78]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:31.618947 5771 leader_election.cc:341] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:31.619333 7211 raft_consensus.cc:482] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:31.619349 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 67327badb2524a2c817045a893bcad6e (DEFAULT_TABLE table=default.Table_faaabef2a0ab42648427c33cda679fb7 [id=7afc114b9ab641df8c540f834036b70c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.619971 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9ea181156f64388ade8a07708b2e814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.619786 7211 raft_consensus.cc:504] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.620222 5335 tablet_service.cc:1467] Processing CreateTablet for tablet 290081bbe4164acd8808041b76441f0d (DEFAULT_TABLE table=default.Table_9936d69a1f7b4853a7c37d5e8da2077f [id=a7e1be7320c34f3bab091c92ec2c1f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.621255 5335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 290081bbe4164acd8808041b76441f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.598848 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet 62d68ab73bae4dd9af46e907b1fd198c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:30 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:31.622804 7211 leader_election.cc:288] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:31.598791 5345 tablet_service.cc:1467] Processing CreateTablet for tablet 67327badb2524a2c817045a893bcad6e (DEFAULT_TABLE table=default.Table_faaabef2a0ab42648427c33cda679fb7 [id=7afc114b9ab641df8c540f834036b70c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.624002 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67327badb2524a2c817045a893bcad6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.600162 5355 tablet_service.cc:1467] Processing CreateTablet for tablet b6b421ff1f094d8689010b4ed6c3f7af (DEFAULT_TABLE table=default.Table_b5830adfb9064ab4b32c29c4c1a03066 [id=84e585111c464794ac7ee0e88227ee7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.627208 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b421ff1f094d8689010b4ed6c3f7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.630541 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 4625b632d67244fc86e48d739dde707e (table default.Table_1665f65d290446e79ac5f6550ec733b7 [id=2f47077a0fd6488ebc33ab8794c0a5b6]) successfully deleted
I20260808 13:35:31.632380 5603 tablet_service.cc:1467] Processing CreateTablet for tablet a9ea181156f64388ade8a07708b2e814 (DEFAULT_TABLE table=default.Table_577d84019e77413d9ff7373a8315d384 [id=6f200cd5eaff4a63a8b2672962db8c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.633414 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9ea181156f64388ade8a07708b2e814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.600576 5332 tablet_service.cc:1467] Processing CreateTablet for tablet c2c50fd06b8841c8b5a944401b495d2e (DEFAULT_TABLE table=default.Table_323ff643b74d4f8e8a82e3fe8d1fc10c [id=4cbe915d8abf43369d194d4087999bc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.637319 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2c50fd06b8841c8b5a944401b495d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.640476 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc9b2e69fe27431dac9eb5ecb9e3e23e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:31.641132 5640 raft_consensus.cc:2454] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:31.643293 6026 ts_tablet_manager.cc:1428] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.077s user 0.002s sys 0.005s
I20260808 13:35:31.647146 6026 tablet_bootstrap.cc:492] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:31.657935 7014 ts_tablet_manager.cc:1918] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:31.678272 7014 log.cc:1196] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/5d388589d386408bb10ca030b87af787
I20260808 13:35:31.679648 7014 ts_tablet_manager.cc:1939] T 5d388589d386408bb10ca030b87af787 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:31.681854 6046 tablet_bootstrap.cc:492] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:31.669801 6026 tablet_bootstrap.cc:654] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:31.682989 6046 ts_tablet_manager.cc:1397] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.388s user 0.008s sys 0.005s
I20260808 13:35:31.684711 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ba43d1479a4f9e9d08bdfa29e7157b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:31.687587 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 262) took 3314 ms. Trace:
I20260808 13:35:31.693833 5600 tablet_service.cc:1467] Processing CreateTablet for tablet 9dee4b3b14264f3aad0896f629ab5b27 (DEFAULT_TABLE table=default.Table_5ede676dcd5843a3b896699541bf6d01 [id=11f8a63441444ac1a7aa7165cbe4b696]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.694202 7014 rpcz_store.cc:276] 0808 13:35:28.373627 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.373747 (+ 120us) service_pool.cc:227] Handling call
0808 13:35:31.529236 (+3155489us) ts_tablet_manager.cc:1145] Deleting tablet 5d388589d386408bb10ca030b87af787
0808 13:35:31.529281 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:31.563107 (+ 33826us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.657722 (+ 94615us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.687528 (+ 29806us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3154909}
I20260808 13:35:31.695005 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dee4b3b14264f3aad0896f629ab5b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.695353 7014 tablet_replica.cc:330] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:31.696036 7014 raft_consensus.cc:2229] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:31.696542 7014 raft_consensus.cc:2258] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:31.688884 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 67327badb2524a2c817045a893bcad6e (DEFAULT_TABLE table=default.Table_faaabef2a0ab42648427c33cda679fb7 [id=7afc114b9ab641df8c540f834036b70c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.690093 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 62d68ab73bae4dd9af46e907b1fd198c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:30 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:31.690757 5611 tablet_service.cc:1467] Processing CreateTablet for tablet b6b421ff1f094d8689010b4ed6c3f7af (DEFAULT_TABLE table=default.Table_b5830adfb9064ab4b32c29c4c1a03066 [id=84e585111c464794ac7ee0e88227ee7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.699545 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67327badb2524a2c817045a893bcad6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.700245 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b421ff1f094d8689010b4ed6c3f7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.701692 6026 tablet_bootstrap.cc:492] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:31.702100 6026 ts_tablet_manager.cc:1397] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.055s user 0.008s sys 0.000s
I20260808 13:35:31.692257 5599 tablet_service.cc:1467] Processing CreateTablet for tablet c2c50fd06b8841c8b5a944401b495d2e (DEFAULT_TABLE table=default.Table_323ff643b74d4f8e8a82e3fe8d1fc10c [id=4cbe915d8abf43369d194d4087999bc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.703718 5599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2c50fd06b8841c8b5a944401b495d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.698312 7014 ts_tablet_manager.cc:1905] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:31.707875 6026 raft_consensus.cc:348] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.711880 6026 raft_consensus.cc:374] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:31.712138 6026 raft_consensus.cc:729] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:31.716514 6026 consensus_queue.cc:260] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.727360 6033 tablet_bootstrap.cc:492] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:31.727702 5832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2c50fd06b8841c8b5a944401b495d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.731635 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b421ff1f094d8689010b4ed6c3f7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.733649 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dee4b3b14264f3aad0896f629ab5b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.735351 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 290081bbe4164acd8808041b76441f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.741043 7013 rpcz_store.cc:276] 0808 13:35:28.227703 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.228049 (+ 346us) service_pool.cc:227] Handling call
0808 13:35:31.360144 (+3132095us) ts_tablet_manager.cc:1145] Deleting tablet 4625b632d67244fc86e48d739dde707e
0808 13:35:31.360200 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:31.480176 (+119976us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.540657 (+ 60481us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:31.579408 (+ 38751us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3131537}
I20260808 13:35:31.743168 7013 tablet_replica.cc:330] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:31.743057 7241 raft_consensus.cc:482] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:31.744009 7013 raft_consensus.cc:2229] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:31.743655 7241 raft_consensus.cc:504] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.746634 7241 leader_election.cc:288] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:31.748348 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1847d0fecb843cdbc833ad674a503a4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:31.749830 5771 leader_election.cc:341] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:31.755012 7199 ts_tablet_manager.cc:1905] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:31.759172 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67327badb2524a2c817045a893bcad6e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:31.757321 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 70041us
I20260808 13:35:31.767912 5606 tablet_service.cc:1467] Processing CreateTablet for tablet 290081bbe4164acd8808041b76441f0d (DEFAULT_TABLE table=default.Table_9936d69a1f7b4853a7c37d5e8da2077f [id=a7e1be7320c34f3bab091c92ec2c1f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:31.769007 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 290081bbe4164acd8808041b76441f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:31.769069 5772 leader_election.cc:302] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:31.771463 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 4625b632d67244fc86e48d739dde707e (table default.Table_1665f65d290446e79ac5f6550ec733b7 [id=2f47077a0fd6488ebc33ab8794c0a5b6]) successfully deleted
I20260808 13:35:31.772614 7387 raft_consensus.cc:2781] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:31.753736 4839 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_95ed126fb41342c1a6a1539f0a5ebbe3 [id=b301b3d4964b41009ed2903dcbefdfae] alter complete (version 1)
I20260808 13:35:31.773453 7387 raft_consensus.cc:482] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:31.686137 6046 raft_consensus.cc:348] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.782369 6046 raft_consensus.cc:374] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:31.782711 6046 raft_consensus.cc:729] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:31.784036 7387 raft_consensus.cc:3037] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:31.784178 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e with delete_type TABLET_DATA_DELETED (replaced by tablet ba39706cff65404485959865fff1772a) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:31.784212 5840 tablet_service.cc:1514] Processing DeleteTablet for tablet e992a0294c404cfb9541b57782a05fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:31.783622 6046 consensus_queue.cc:260] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:31.786906 4839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 34) took 1841 ms (client timeout 1000 ms). Trace:
W20260808 13:35:31.787302 4839 rpcz_store.cc:269] 0808 13:35:29.945362 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.235097 (+289735us) service_pool.cc:227] Handling call
0808 13:35:30.276794 (+ 41697us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:31.786824 (+1510030us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:30.396437 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:30.396814 (+ 377us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:30.396841 (+ 27us) write_op.cc:435] Acquired schema lock
0808 13:35:30.396853 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:30.397106 (+ 253us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:30.397141 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:30.397166 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:30.397316 (+ 150us) tablet.cc:699] Row locks acquired
0808 13:35:30.397330 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:30.397452 (+ 122us) write_op.cc:270] Start()
0808 13:35:30.397531 (+ 79us) write_op.cc:276] Timestamp: P: 1786196130396627 usec, L: 0
0808 13:35:30.397546 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:30.490373 (+ 92827us) log.cc:842] Serialized 356 byte log entry
0808 13:35:30.499531 (+ 9158us) spinlock_profiling.cc:243] Waited 92.3 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:30.526533 (+ 27002us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b2000001078. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c20f1f5 00007f227c22b1b1 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784
0808 13:35:30.528417 (+ 1884us) op_driver.cc:444] REPLICATION: finished
0808 13:35:30.530879 (+ 2462us) write_op.cc:301] APPLY: starting
0808 13:35:30.530940 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:30.531087 (+ 147us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:30.531122 (+ 35us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:30.531349 (+ 227us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:30.531547 (+ 198us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:30.531612 (+ 65us) write_op.cc:312] APPLY: finished
0808 13:35:30.531862 (+ 250us) log.cc:842] Serialized 23 byte log entry
0808 13:35:30.532067 (+ 205us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:30.532148 (+ 81us) write_op.cc:454] Released schema lock
0808 13:35:30.532221 (+ 73us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:30.532471 (+ 250us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:30.532511 (+ 40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19094,"reported_tablets":8,"spinlock_wait_cycles":75939072,"child_traces":[["op",{"apply.queue_time_us":2211,"apply.run_cpu_time_us":1840,"apply.run_wall_time_us":1841,"num_ops":1,"prepare.queue_time_us":120,"prepare.run_cpu_time_us":5653,"prepare.run_wall_time_us":130211,"raft.queue_time_us":2069,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":344,"replication_time_us":130766,"spinlock_wait_cycles":272151168}]]}
I20260808 13:35:31.809031 6033 tablet_bootstrap.cc:654] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:31.814941 4838 master_service.cc:946] ReplaceTablet: received request to replace tablet 8d12106f0d794a79935999a2b8dbe7e9 from {username='slave'} at 127.0.0.1:60270
W20260808 13:35:31.826788 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 57818.8us
I20260808 13:35:31.828838 7241 consensus_queue.cc:1035] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:31.831284 5283 leader_election.cc:341] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:31.834824 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_7cfe2eb0d78341a0816f9b5e14316b4f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:31.836766 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7cfe2eb0d78341a0816f9b5e14316b4f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:31.850814 4837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d553a5c837764412ab01800bea0017c2" } modify_external_catalogs: true
I20260808 13:35:31.851320 4837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d553a5c837764412ab01800bea0017c2" } modify_external_catalogs: true
I20260808 13:35:31.854991 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_feb9646b81ef46c7aa00ea076ae7240f" } modify_external_catalogs: true
I20260808 13:35:31.855432 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_3585f6d6de0e451e9732239606d0e39c" } new_table_name: "default.Table_5128302bae8343618fb6509466b51c71" modify_external_catalogs: true
I20260808 13:35:31.856066 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_41a67a20ea754c6fb307fb434f9711ae" } new_table_name: "default.Table_510c442cafba44a9bf48a292ca01fbd4" modify_external_catalogs: true
I20260808 13:35:31.857722 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_feb9646b81ef46c7aa00ea076ae7240f" } modify_external_catalogs: true
I20260808 13:35:31.860467 7013 raft_consensus.cc:2258] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:31.864383 7013 ts_tablet_manager.cc:1905] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:31.871091 6450 hybrid_clock.cc:648] HybridClock initialized: now 1786196131870986 us; error 102 us; skew 500 ppm
I20260808 13:35:31.872581 6450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:35:31.887648 6026 ts_tablet_manager.cc:1428] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.185s user 0.005s sys 0.016s
I20260808 13:35:31.867272 7211 raft_consensus.cc:482] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:31.902915 7211 raft_consensus.cc:504] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.912611 7211 leader_election.cc:288] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:31.913519 6901 raft_consensus.cc:482] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:31.914019 6901 raft_consensus.cc:504] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:31.872445 6046 ts_tablet_manager.cc:1428] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.189s user 0.007s sys 0.000s
I20260808 13:35:31.922353 6046 tablet_bootstrap.cc:492] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:31.946245 7435 raft_consensus.cc:482] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:31.947232 7435 raft_consensus.cc:504] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:31.949333 5608 consensus_meta.cc:287] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.538s user 0.004s sys 0.006s
I20260808 13:35:31.964030 7435 leader_election.cc:288] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:31.967612 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba098c946794607b2a15bc64098d918" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:31.969987 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16a6552c871b4b81b6520cb084d359a2" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:31.969857 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba098c946794607b2a15bc64098d918" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:31.971426 5551 leader_election.cc:341] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:31.971801 5550 leader_election.cc:341] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:31.972227 5551 leader_election.cc:341] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:31.972638 5551 leader_election.cc:302] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:31.976827 6033 tablet_bootstrap.cc:492] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:31.977288 6033 ts_tablet_manager.cc:1397] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.250s user 0.013s sys 0.004s
I20260808 13:35:31.980906 6033 raft_consensus.cc:348] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:31.981590 6033 raft_consensus.cc:374] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:31.981909 6033 raft_consensus.cc:729] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:31.983773 6026 tablet_bootstrap.cc:492] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:31.987600 7460 tablet.cc:1722] T 5c3535f396f74749a6a28907c6d93abd P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:31.989388 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ba43d1479a4f9e9d08bdfa29e7157b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:31.990499 5640 raft_consensus.cc:2454] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:31.996536 6046 tablet_bootstrap.cc:654] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:31.997022 4836 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_7fd510a91deb449091ff7da8c644fae1 [id=1b86c1f240214c88b29348f33a11503d] alter complete (version 1)
W20260808 13:35:31.999248 4836 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 44) took 1368 ms (client timeout 1000 ms). Trace:
I20260808 13:35:31.982594 6033 consensus_queue.cc:260] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.001426 7013 ts_tablet_manager.cc:1918] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:32.001820 7013 log.cc:1196] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/5d388589d386408bb10ca030b87af787
W20260808 13:35:31.999550 4836 rpcz_store.cc:269] 0808 13:35:30.630457 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.630571 (+ 114us) service_pool.cc:227] Handling call
0808 13:35:31.999175 (+1368604us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:31.777428 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:31.777815 (+ 387us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:31.777838 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:31.777852 (+ 14us) tablet.cc:660] Decoding operations
0808 13:35:31.778112 (+ 260us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:31.778151 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:31.778173 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:31.778318 (+ 145us) tablet.cc:699] Row locks acquired
0808 13:35:31.778332 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:31.778434 (+ 102us) write_op.cc:270] Start()
0808 13:35:31.778518 (+ 84us) write_op.cc:276] Timestamp: P: 1786196131778341 usec, L: 0
0808 13:35:31.778535 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:31.839177 (+ 60642us) log.cc:842] Serialized 356 byte log entry
0808 13:35:31.840570 (+ 1393us) spinlock_profiling.cc:243] Waited 60.1 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:31.994871 (+154301us) op_driver.cc:444] REPLICATION: finished
0808 13:35:31.995222 (+ 351us) write_op.cc:301] APPLY: starting
0808 13:35:31.995320 (+ 98us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:31.995427 (+ 107us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:31.995451 (+ 24us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:31.995625 (+ 174us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:31.995777 (+ 152us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:31.995830 (+ 53us) write_op.cc:312] APPLY: finished
0808 13:35:31.996175 (+ 345us) log.cc:842] Serialized 23 byte log entry
0808 13:35:31.996439 (+ 264us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:31.996521 (+ 82us) write_op.cc:454] Released schema lock
0808 13:35:31.996641 (+ 120us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":835,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":145,"num_ops":1,"prepare.queue_time_us":103,"prepare.run_cpu_time_us":3929,"prepare.run_wall_time_us":63353,"raft.queue_time_us":96,"raft.run_cpu_time_us":805,"raft.run_wall_time_us":39848,"replication_time_us":216209,"spinlock_wait_cycles":138228480}]]}
I20260808 13:35:32.006749 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c23b9f32680f412fbc697e136022e69c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:32.010057 6033 ts_tablet_manager.cc:1428] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.032s user 0.008s sys 0.000s
I20260808 13:35:32.011168 6033 tablet_bootstrap.cc:492] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:32.013440 6450 webserver.cc:460] Webserver started at http://127.4.170.60:45565/ using document root <none> and password file <none>
I20260808 13:35:32.015465 6450 fs_manager.cc:359] Metadata directory not provided
I20260808 13:35:32.015936 6450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:35:32.019546 7013 ts_tablet_manager.cc:1939] T 5d388589d386408bb10ca030b87af787 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:32.027644 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 272) took 3721 ms. Trace:
W20260808 13:35:32.028121 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c23b9f32680f412fbc697e136022e69c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:32.031153 4823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111) [suppressed 4 similar messages]
I20260808 13:35:32.030745 7013 rpcz_store.cc:276] 0808 13:35:28.306593 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.310732 (+ 4139us) service_pool.cc:227] Handling call
0808 13:35:31.743003 (+3432271us) ts_tablet_manager.cc:1145] Deleting tablet 5d388589d386408bb10ca030b87af787
0808 13:35:31.743054 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:31.938169 (+195115us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.001180 (+ 63011us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.027578 (+ 26398us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3431512}
I20260808 13:35:32.032305 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 4625b632d67244fc86e48d739dde707e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:27 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:32.034312 7013 tablet_replica.cc:330] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:32.035655 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 5d388589d386408bb10ca030b87af787 (table default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88]) successfully deleted
I20260808 13:35:31.976348 7387 raft_consensus.cc:504] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.034955 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f2e0b98af431d99894e8778257bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:32.038482 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d388589d386408bb10ca030b87af787 with delete_type TABLET_DATA_DELETED (replaced by tablet 3ab26fa1872e4acf874f6577bded5e43) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:32.039794 7013 raft_consensus.cc:2229] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
W20260808 13:35:32.047575 4823 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5663fa6a1bb148c3afc9340477b05a4b consensus state has no leader
I20260808 13:35:32.040347 7387 leader_election.cc:288] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:32.048081 7013 raft_consensus.cc:2258] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:32.054134 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet e992a0294c404cfb9541b57782a05fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:32.055228 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet a541ec35b1a44109866fabc953f14cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:13 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:32.057042 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 896389941179456d9d0ed3cd14726f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:32.058764 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e7f2daaad4f48a79b94344ff35e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:10 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:32.059850 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c59eea10ec45af8e3a0992f9cc74bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:11 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:32.067324 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 50) took 1633 ms (client timeout 1000 ms). Trace:
I20260808 13:35:32.080518 7013 ts_tablet_manager.cc:1905] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:32.082763 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd6945ecccc4684a3fa5b2a1f07057f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:32.084144 5637 raft_consensus.cc:2379] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:32.095587 6046 tablet_bootstrap.cc:492] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:32.097775 6046 ts_tablet_manager.cc:1397] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.176s user 0.010s sys 0.015s
I20260808 13:35:32.078817 7435 raft_consensus.cc:2726] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:32.104866 5285 consensus_peers.cc:487] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 896389941179456d9d0ed3cd14726f11. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:32.107481 7211 consensus_queue.cc:1035] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:32.109771 5285 leader_election.cc:302] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:32.110738 7211 raft_consensus.cc:2781] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:32.078205 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 8d12106f0d794a79935999a2b8dbe7e9
I20260808 13:35:32.112479 4838 catalog_manager.cc:6565] ReplaceTablet: tablet 8d12106f0d794a79935999a2b8dbe7e9 deleted and replaced by tablet 30691adffb1f40bdaeaa24f978a38bd7
I20260808 13:35:32.106297 6046 raft_consensus.cc:348] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.113376 6046 raft_consensus.cc:374] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:32.113662 6046 raft_consensus.cc:729] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:32.084576 4837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be
I20260808 13:35:32.114238 6046 consensus_queue.cc:260] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.122000 6046 ts_tablet_manager.cc:1428] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260808 13:35:32.095753 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_ec97d7d5adc245a6b9d4224e8ab75cd5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:32.122908 6046 tablet_bootstrap.cc:492] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:32.123628 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec97d7d5adc245a6b9d4224e8ab75cd5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:32.103922 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 5d388589d386408bb10ca030b87af787 (table default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88]) successfully deleted
I20260808 13:35:32.111052 7211 raft_consensus.cc:482] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:32.125583 7211 raft_consensus.cc:3037] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:32.129283 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d12106f0d794a79935999a2b8dbe7e9 with delete_type TABLET_DATA_DELETED (replaced by tablet 30691adffb1f40bdaeaa24f978a38bd7) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:32.130100 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:31 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:32.133258 4835 rpcz_store.cc:269] 0808 13:35:30.433868 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.434004 (+ 136us) service_pool.cc:227] Handling call
0808 13:35:30.493176 (+ 59172us) spinlock_profiling.cc:243] Waited 46.2 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:32.067190 (+1574014us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3868,"reported_tablets":27,"spinlock_wait_cycles":106312448}
I20260808 13:35:32.080920 6901 leader_election.cc:288] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:32.153141 7454 tablet.cc:1722] T 5c3535f396f74749a6a28907c6d93abd P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:32.154755 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc9b2e69fe27431dac9eb5ecb9e3e23e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:32.160022 5771 leader_election.cc:341] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:32.163287 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 69642fdba1f145438b04eae9f8154101 with delete_type TABLET_DATA_DELETED (replaced by tablet 538d9a883e9d46668fde30876352d20e) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:32.164995 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d12106f0d794a79935999a2b8dbe7e9 with delete_type TABLET_DATA_DELETED (replaced by tablet 30691adffb1f40bdaeaa24f978a38bd7) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:32.165719 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:31 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:32.170223 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_30ba5bac739643b2846759c8994126bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:32.172027 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30ba5bac739643b2846759c8994126bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:32.193253 5833 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:31 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:32.195365 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d12106f0d794a79935999a2b8dbe7e9 with delete_type TABLET_DATA_DELETED (replaced by tablet 30691adffb1f40bdaeaa24f978a38bd7) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:32.198531 6026 tablet_bootstrap.cc:654] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:32.213162 6033 tablet_bootstrap.cc:654] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:32.201475 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd6945ecccc4684a3fa5b2a1f07057f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:32.220546 7241 raft_consensus.cc:482] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 946c9d8462644deba5b45fa724f1923c)
I20260808 13:35:32.222270 7241 raft_consensus.cc:504] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.241938 6046 tablet_bootstrap.cc:654] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:32.245674 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 249323us
I20260808 13:35:32.261724 6450 fs_manager.cc:709] Time spent opening directory manager: real 0.211s user 0.003s sys 0.006s
W20260808 13:35:32.263113 5771 leader_election.cc:341] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Not found: Tablet not found: 9260370cf8cb46a7b699581e4e8d46db
I20260808 13:35:32.266469 7241 leader_election.cc:288] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:32.267900 7241 raft_consensus.cc:482] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:32.268328 7241 raft_consensus.cc:504] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.270292 7241 leader_election.cc:288] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:32.276283 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f2e0b98af431d99894e8778257bc4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:32.277498 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1847d0fecb843cdbc833ad674a503a4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:32.284789 7211 raft_consensus.cc:504] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.294050 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc9b2e69fe27431dac9eb5ecb9e3e23e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:32.294649 5634 raft_consensus.cc:3037] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:32.304361 4833 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ffbdf03ebe4e48a98450e90a3a78181c consensus state has no leader
I20260808 13:35:32.305284 5634 raft_consensus.cc:2454] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:32.306742 4833 master_service.cc:946] ReplaceTablet: received request to replace tablet 1ba5fa465ec04bbb801a821a71a8f87d from {username='slave'} at 127.0.0.1:60270
W20260808 13:35:32.314196 4784 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 97423.2us
I20260808 13:35:32.315239 7485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:35:32.316634 6450 fs_manager.cc:725] Time spent opening block manager: real 0.051s user 0.006s sys 0.000s
I20260808 13:35:32.317066 6450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal
uuid: "87e8b888041b4135af55792050defcb5"
format_stamp: "Formatted at 2026-08-08 13:34:47 on dist-test-slave-twwt"
I20260808 13:35:32.321621 6450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:35:32.322999 4841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 20113491b108476b95b8f32b69176d0e
I20260808 13:35:32.326694 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 20113491b108476b95b8f32b69176d0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:31 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:32.327752 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 162203us
W20260808 13:35:32.333954 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 42e36d643850411498c23699835f55a6 consensus state has no leader
I20260808 13:35:32.303707 5383 raft_consensus.cc:3037] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:32.337385 6033 tablet_bootstrap.cc:492] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:32.325739 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 20113491b108476b95b8f32b69176d0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:31 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:32.337924 6033 ts_tablet_manager.cc:1397] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.327s user 0.008s sys 0.009s
W20260808 13:35:32.339112 5551 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 45199.5us
I20260808 13:35:32.341277 6046 tablet_bootstrap.cc:492] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:32.341696 6046 ts_tablet_manager.cc:1397] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.219s user 0.009s sys 0.000s
I20260808 13:35:32.342880 5599 tablet_service.cc:1514] Processing DeleteTablet for tablet 20113491b108476b95b8f32b69176d0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:31 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:32.347263 7473 consensus_queue.cc:1035] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:32.350241 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ba43d1479a4f9e9d08bdfa29e7157b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:32.359216 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_da0f5516539d4f17b7ad03013cfd9460"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:32.360549 5637 raft_consensus.cc:3037] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:32.360848 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da0f5516539d4f17b7ad03013cfd9460 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:32.361179 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f2e0b98af431d99894e8778257bc4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:32.364243 7211 leader_election.cc:288] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:32.364692 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6529b65492c94717b809b0a0c5f23509" } modify_external_catalogs: true
I20260808 13:35:32.365237 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6529b65492c94717b809b0a0c5f23509" } modify_external_catalogs: true
I20260808 13:35:32.387238 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ba43d1479a4f9e9d08bdfa29e7157b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:32.391985 6026 tablet_bootstrap.cc:492] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:32.392452 6026 ts_tablet_manager.cc:1397] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.409s user 0.011s sys 0.007s
I20260808 13:35:32.392352 6033 raft_consensus.cc:348] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.393319 6033 raft_consensus.cc:374] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:32.393633 6033 raft_consensus.cc:729] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:32.364082 6046 raft_consensus.cc:348] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.402776 6046 raft_consensus.cc:374] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:32.403074 6046 raft_consensus.cc:729] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:32.404353 6046 consensus_queue.cc:260] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.406929 6046 ts_tablet_manager.cc:1428] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.065s user 0.008s sys 0.000s
I20260808 13:35:32.407785 6046 tablet_bootstrap.cc:492] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:32.419441 6026 raft_consensus.cc:348] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.421443 6026 raft_consensus.cc:374] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:32.421795 6026 raft_consensus.cc:729] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:32.402017 7387 consensus_queue.cc:1035] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:32.424691 7014 ts_tablet_manager.cc:1918] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:32.425091 7014 log.cc:1196] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/a8e18e51c2e24d3caa3d29e6645e46ae
I20260808 13:35:32.425863 7014 ts_tablet_manager.cc:1939] T a8e18e51c2e24d3caa3d29e6645e46ae P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:32.425923 5283 leader_election.cc:302] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:32.426888 6033 consensus_queue.cc:260] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:32.414671 5772 leader_election.cc:341] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:32.327790 7199 ts_tablet_manager.cc:1918] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:32.436076 7199 log.cc:1196] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/6826e4e4db2043b3a4fe9e7a9359cab6
I20260808 13:35:32.437111 7199 ts_tablet_manager.cc:1939] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:32.440037 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 262) took 4363 ms. Trace:
I20260808 13:35:32.440841 7346 raft_consensus.cc:3037] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:32.442466 7199 rpcz_store.cc:276] 0808 13:35:28.076826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.076977 (+ 151us) service_pool.cc:227] Handling call
0808 13:35:31.425839 (+3348862us) ts_tablet_manager.cc:1145] Deleting tablet 6826e4e4db2043b3a4fe9e7a9359cab6
0808 13:35:31.539497 (+113658us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:32.010453 (+470956us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.327391 (+316938us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.439970 (+112579us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3335783}
I20260808 13:35:32.444459 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 6826e4e4db2043b3a4fe9e7a9359cab6 (table default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]) successfully deleted
I20260808 13:35:32.445273 5772 leader_election.cc:302] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:32.447745 7387 raft_consensus.cc:2726] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:32.451853 7199 tablet_replica.cc:330] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:32.456490 6026 consensus_queue.cc:260] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.461665 7013 ts_tablet_manager.cc:1918] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:32.455084 6033 ts_tablet_manager.cc:1428] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.117s user 0.002s sys 0.007s
I20260808 13:35:32.457916 7476 consensus_queue.cc:1035] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:32.481107 5637 raft_consensus.cc:2454] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:32.462051 7013 log.cc:1196] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/a8e18e51c2e24d3caa3d29e6645e46ae
I20260808 13:35:32.459167 7387 raft_consensus.cc:482] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:32.460245 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16a6552c871b4b81b6520cb084d359a2" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:32.483381 5285 leader_election.cc:302] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:32.482833 7387 raft_consensus.cc:504] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.484175 7013 ts_tablet_manager.cc:1939] T a8e18e51c2e24d3caa3d29e6645e46ae P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:32.485875 7387 leader_election.cc:288] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:32.485363 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 273) took 4009 ms. Trace:
I20260808 13:35:32.487116 7199 raft_consensus.cc:2229] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:32.487450 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69642fdba1f145438b04eae9f8154101" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:32.488802 7491 raft_consensus.cc:2781] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:32.491230 7199 raft_consensus.cc:2258] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:32.486742 7013 rpcz_store.cc:276] 0808 13:35:28.475623 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.475740 (+ 117us) service_pool.cc:227] Handling call
0808 13:35:32.034107 (+3558367us) ts_tablet_manager.cc:1145] Deleting tablet a8e18e51c2e24d3caa3d29e6645e46ae
0808 13:35:32.034149 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:32.272951 (+238802us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.461432 (+188481us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.485309 (+ 23877us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3557687}
I20260808 13:35:32.473446 5772 leader_election.cc:302] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:32.488837 5771 leader_election.cc:341] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:32.496512 7387 raft_consensus.cc:2781] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:32.497255 7387 raft_consensus.cc:686] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:32.498164 7387 consensus_queue.cc:237] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.488265 6033 tablet_bootstrap.cc:492] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:32.501148 5550 leader_election.cc:341] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:32.501612 5550 leader_election.cc:302] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:32.502791 7435 raft_consensus.cc:2726] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:32.506019 6026 ts_tablet_manager.cc:1428] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.113s user 0.007s sys 0.002s
I20260808 13:35:32.491226 7491 raft_consensus.cc:686] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:32.507527 7491 consensus_queue.cc:237] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:32.496065 5772 leader_election.cc:341] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 9260370cf8cb46a7b699581e4e8d46db
I20260808 13:35:32.511880 5772 leader_election.cc:302] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:32.505815 7500 tablet.cc:1722] T 5c3535f396f74749a6a28907c6d93abd P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:32.513201 7476 raft_consensus.cc:2726] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:32.513406 5772 leader_election.cc:302] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:32.521379 7199 ts_tablet_manager.cc:1905] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:32.524852 5283 leader_election.cc:341] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:32.529666 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet a8e18e51c2e24d3caa3d29e6645e46ae (table default.Table_321b4eb3dd024925af1cb354d3fe1567 [id=3c108c9e89424d9a9e11554601cf427c]) successfully deleted
I20260808 13:35:32.530848 7013 tablet_replica.cc:330] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:32.532970 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 263) took 4110 ms. Trace:
I20260808 13:35:32.536011 7013 raft_consensus.cc:2229] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:32.539665 6026 tablet_bootstrap.cc:492] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:32.540640 7013 raft_consensus.cc:2258] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:32.502030 7346 raft_consensus.cc:2726] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:32.522058 7476 raft_consensus.cc:2726] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:32.548341 7241 consensus_queue.cc:1035] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:32.551764 7489 raft_consensus.cc:482] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 98dd96f425fc4ab3a6a54f2dfaaf9fab)
I20260808 13:35:32.552345 7489 raft_consensus.cc:504] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:32.556612 5285 leader_election.cc:341] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: d3682c9c656c4556bc13ae3aeddd7c5e
I20260808 13:35:32.533290 7014 rpcz_store.cc:276] 0808 13:35:28.422524 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.422687 (+ 163us) service_pool.cc:227] Handling call
0808 13:35:31.695193 (+3272506us) ts_tablet_manager.cc:1145] Deleting tablet a8e18e51c2e24d3caa3d29e6645e46ae
0808 13:35:31.695249 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:32.152984 (+457735us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.424446 (+271462us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.532905 (+108459us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3271976}
I20260808 13:35:32.557274 7489 leader_election.cc:288] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:32.562709 7241 raft_consensus.cc:482] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5cd987c3dbdb4168beb73cd7e577c7cb)
I20260808 13:35:32.563210 7241 raft_consensus.cc:504] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.563939 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69642fdba1f145438b04eae9f8154101" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:32.564620 5636 raft_consensus.cc:2454] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:32.564993 7241 leader_election.cc:288] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:32.571089 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet a8e18e51c2e24d3caa3d29e6645e46ae (table default.Table_321b4eb3dd024925af1cb354d3fe1567 [id=3c108c9e89424d9a9e11554601cf427c]) successfully deleted
W20260808 13:35:32.571187 5771 leader_election.cc:341] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Not found: Tablet not found: 4625b632d67244fc86e48d739dde707e
I20260808 13:35:32.571978 7014 tablet_replica.cc:330] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:32.575449 6033 tablet_bootstrap.cc:654] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:32.572767 7014 raft_consensus.cc:2229] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:32.576763 7014 raft_consensus.cc:2258] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:32.505927 6046 tablet_bootstrap.cc:654] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:32.592080 5882 raft_consensus.cc:3037] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:32.597674 4837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_30ba5bac739643b2846759c8994126bc" } modify_external_catalogs: true
I20260808 13:35:32.598274 4837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_30ba5bac739643b2846759c8994126bc" } modify_external_catalogs: true
I20260808 13:35:32.602707 7013 ts_tablet_manager.cc:1905] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:32.603988 4835 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c4e53d8c393449429ef2791497ee7208" } new_table_name: "default.Table_07a60e9e6b984fbd97ce35f3f18b895a" modify_external_catalogs: true
I20260808 13:35:32.607446 4836 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_e0361960d323473f83c14c853ffcb172" } new_table_name: "default.Table_41f3478fe8c94fbfaf7fcbdee22ce987" modify_external_catalogs: true
I20260808 13:35:32.615468 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_dfee9bf8bf9e4bab8982be9ddb880b00"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:32.616953 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dfee9bf8bf9e4bab8982be9ddb880b00 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:32.620462 5283 leader_election.cc:341] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: d3682c9c656c4556bc13ae3aeddd7c5e
I20260808 13:35:32.620846 5283 leader_election.cc:302] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:32.621958 7346 raft_consensus.cc:2726] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:32.624013 5772 leader_election.cc:302] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:32.624672 7473 raft_consensus.cc:2781] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:32.625057 7473 raft_consensus.cc:482] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:32.625433 7473 raft_consensus.cc:3037] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:32.633509 6046 tablet_bootstrap.cc:492] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:32.633919 6046 ts_tablet_manager.cc:1397] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.226s user 0.009s sys 0.000s
I20260808 13:35:32.649336 7473 raft_consensus.cc:504] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.650214 4837 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet aec3374799504d9f865d15bd9eed9d56
I20260808 13:35:32.652197 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69642fdba1f145438b04eae9f8154101" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:32.654641 7473 leader_election.cc:288] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:32.655985 7473 consensus_queue.cc:1035] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:32.656387 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_0ab0a38241284050a12358d0f8d76032"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:32.658550 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ab0a38241284050a12358d0f8d76032 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:32.664758 6033 tablet_bootstrap.cc:492] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:32.665496 6033 ts_tablet_manager.cc:1397] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.177s user 0.011s sys 0.000s
I20260808 13:35:32.667129 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69642fdba1f145438b04eae9f8154101" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:32.667721 5640 raft_consensus.cc:3037] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:32.667936 6033 raft_consensus.cc:348] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.668720 6033 raft_consensus.cc:374] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:32.668954 6033 raft_consensus.cc:729] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:32.669777 6033 consensus_queue.cc:260] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.671965 6033 ts_tablet_manager.cc:1428] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 13:35:32.694419 7014 ts_tablet_manager.cc:1905] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:32.701076 4835 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4b7284afe0294644a368cf51451c7ab6 consensus state has no leader
I20260808 13:35:32.706038 6026 tablet_bootstrap.cc:654] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:32.708572 5640 raft_consensus.cc:2454] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:32.712371 7241 consensus_queue.cc:1035] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:35:32.713212 5772 leader_election.cc:341] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 4625b632d67244fc86e48d739dde707e
I20260808 13:35:32.717295 5772 leader_election.cc:302] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:32.718025 7241 raft_consensus.cc:2726] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:32.725147 7199 ts_tablet_manager.cc:1918] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:32.725931 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_fbaf65bcf1fd48249596df8d79af088b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:32.727516 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fbaf65bcf1fd48249596df8d79af088b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:32.730391 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_bc6082a17c754033966ecce7c4f9bf26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:32.731020 7199 log.cc:1196] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/896389941179456d9d0ed3cd14726f11
I20260808 13:35:32.731786 7199 ts_tablet_manager.cc:1939] T 896389941179456d9d0ed3cd14726f11 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
W20260808 13:35:32.731801 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc6082a17c754033966ecce7c4f9bf26 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:32.725847 5772 leader_election.cc:302] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:32.730312 6033 tablet_bootstrap.cc:492] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:32.735021 7387 raft_consensus.cc:2781] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:32.736230 7387 raft_consensus.cc:686] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:32.745859 6033 tablet_bootstrap.cc:654] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:32.754256 6046 raft_consensus.cc:348] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.755103 6046 raft_consensus.cc:374] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:32.755410 6046 raft_consensus.cc:729] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:32.756037 6046 consensus_queue.cc:260] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.737157 7387 consensus_queue.cc:237] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:32.757807 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 261) took 4682 ms. Trace:
I20260808 13:35:32.763644 7199 rpcz_store.cc:276] 0808 13:35:28.075332 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.075518 (+ 186us) service_pool.cc:227] Handling call
0808 13:35:32.451589 (+4376071us) ts_tablet_manager.cc:1145] Deleting tablet 896389941179456d9d0ed3cd14726f11
0808 13:35:32.451680 (+ 91us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:32.636971 (+185291us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.724924 (+ 87953us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.757735 (+ 32811us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4360947}
I20260808 13:35:32.764324 6026 tablet_bootstrap.cc:492] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:32.764816 6026 ts_tablet_manager.cc:1397] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.225s user 0.005s sys 0.005s
I20260808 13:35:32.765961 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 256) took 4739 ms. Trace:
I20260808 13:35:32.773407 7199 rpcz_store.cc:276] 0808 13:35:28.026240 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.026875 (+ 635us) service_pool.cc:227] Handling call
0808 13:35:32.765670 (+4738795us) ts_tablet_manager.cc:1145] Deleting tablet 06c59eea10ec45af8e3a0992f9cc74bf
0808 13:35:32.765718 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:32.765910 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4674775}
I20260808 13:35:32.778815 6026 raft_consensus.cc:348] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.779507 6026 raft_consensus.cc:374] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:32.779754 6026 raft_consensus.cc:729] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:32.788148 6026 consensus_queue.cc:260] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.801162 6026 ts_tablet_manager.cc:1428] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.036s user 0.006s sys 0.001s
I20260808 13:35:32.807392 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 260) took 4765 ms. Trace:
I20260808 13:35:32.807734 7199 rpcz_store.cc:276] 0808 13:35:28.041966 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.042136 (+ 170us) service_pool.cc:227] Handling call
0808 13:35:32.807039 (+4764903us) ts_tablet_manager.cc:1145] Deleting tablet f0c76b819474468baac51365360becfc
0808 13:35:32.807110 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:32.807331 (+ 221us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4715582}
I20260808 13:35:32.808931 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 264) took 4700 ms. Trace:
I20260808 13:35:32.809217 7199 rpcz_store.cc:276] 0808 13:35:28.108750 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.117690 (+ 8940us) service_pool.cc:227] Handling call
0808 13:35:32.808545 (+4690855us) ts_tablet_manager.cc:1145] Deleting tablet 3424ca753bcc4f448d314c56aba3323e
0808 13:35:32.808594 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:32.808884 (+ 290us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4690279}
I20260808 13:35:32.810232 7199 tablet_replica.cc:330] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:32.814010 6026 tablet_bootstrap.cc:492] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:32.823258 7014 ts_tablet_manager.cc:1918] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:32.826815 7199 raft_consensus.cc:2229] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:32.827407 7199 raft_consensus.cc:2258] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:32.823623 7014 log.cc:1196] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/e00f2e0b98af431d99894e8778257bc4
I20260808 13:35:32.833384 7014 ts_tablet_manager.cc:1939] T e00f2e0b98af431d99894e8778257bc4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:32.837011 7199 ts_tablet_manager.cc:1905] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:32.846984 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 896389941179456d9d0ed3cd14726f11 (table default.Table_41272f24d0a5431b9f718128ad86cb81 [id=d7c7a7f08fa543aaa0bca2e996d1ee8e]) successfully deleted
W20260808 13:35:32.851698 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 06c59eea10ec45af8e3a0992f9cc74bf because the tablet was not found. No further retry: Not found: Tablet not found: 06c59eea10ec45af8e3a0992f9cc74bf
W20260808 13:35:32.852618 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet f0c76b819474468baac51365360becfc because the tablet was not found. No further retry: Not found: Tablet not found: f0c76b819474468baac51365360becfc
I20260808 13:35:32.854676 4837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a7a4e8c009fd4046ad8a17c30d9c4de5" } modify_external_catalogs: true
I20260808 13:35:32.855700 4837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a7a4e8c009fd4046ad8a17c30d9c4de5" } modify_external_catalogs: true
I20260808 13:35:32.864208 6033 tablet_bootstrap.cc:492] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:32.871942 6033 ts_tablet_manager.cc:1397] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.145s user 0.014s sys 0.006s
I20260808 13:35:32.875273 6033 raft_consensus.cc:348] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.879148 6033 raft_consensus.cc:374] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:32.879787 6033 raft_consensus.cc:729] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:32.880893 6033 consensus_queue.cc:260] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.888636 6033 ts_tablet_manager.cc:1428] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260808 13:35:32.892793 6033 tablet_bootstrap.cc:492] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:32.901090 6033 tablet_bootstrap.cc:654] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:32.907222 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_bbaf205810de458492573e8be388ece0" } new_table_name: "default.Table_f3144416bf1140aaaa78d9ec41319c6b" modify_external_catalogs: true
I20260808 13:35:32.916498 7434 raft_consensus.cc:482] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:32.916723 6026 tablet_bootstrap.cc:654] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:32.916997 7434 raft_consensus.cc:504] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.918695 6033 tablet_bootstrap.cc:492] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:32.919165 7434 leader_election.cc:288] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:32.920094 6033 ts_tablet_manager.cc:1397] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.002s
I20260808 13:35:32.921841 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8514ca242bb2418d8ddbb88548b21832" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:32.922410 5882 raft_consensus.cc:2454] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:32.922515 6033 raft_consensus.cc:348] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.923324 6033 raft_consensus.cc:374] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:32.923624 6033 raft_consensus.cc:729] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:32.923704 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 265) took 4412 ms. Trace:
I20260808 13:35:32.923673 5550 leader_election.cc:302] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:32.925782 7434 raft_consensus.cc:2781] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:32.926158 7434 raft_consensus.cc:482] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:32.926651 7434 raft_consensus.cc:3037] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:32.930459 6033 consensus_queue.cc:260] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.924145 7014 rpcz_store.cc:276] 0808 13:35:28.511630 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.511762 (+ 132us) service_pool.cc:227] Handling call
0808 13:35:32.571808 (+4060046us) ts_tablet_manager.cc:1145] Deleting tablet e00f2e0b98af431d99894e8778257bc4
0808 13:35:32.571855 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:32.757930 (+186075us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.823027 (+ 65097us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.923639 (+100612us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4059429}
I20260808 13:35:32.936807 7014 tablet_replica.cc:330] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:32.939481 6033 ts_tablet_manager.cc:1428] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260808 13:35:32.939225 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8514ca242bb2418d8ddbb88548b21832" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:32.940917 6033 tablet_bootstrap.cc:492] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:32.941332 7014 raft_consensus.cc:2229] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:32.941772 7014 raft_consensus.cc:2258] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:32.941919 5551 leader_election.cc:341] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:32.857461 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 3424ca753bcc4f448d314c56aba3323e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3424ca753bcc4f448d314c56aba3323e already in progress: creating tablet
W20260808 13:35:32.945004 4819 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 87658.7us
I20260808 13:35:32.945415 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet e00f2e0b98af431d99894e8778257bc4 (table default.Table_8523d4a015194c64b22f66a22b0dcc06 [id=b285a956530e4fb4a5d1632a45d87c0e]) successfully deleted
I20260808 13:35:32.948104 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_01763d5cef124bad885eaaba0686787b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:32.951089 6046 ts_tablet_manager.cc:1428] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.317s user 0.011s sys 0.004s
I20260808 13:35:32.953486 6046 tablet_bootstrap.cc:492] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:32.961952 7199 ts_tablet_manager.cc:1918] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:32.962355 7199 log.cc:1196] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/9260370cf8cb46a7b699581e4e8d46db
I20260808 13:35:32.963999 7199 ts_tablet_manager.cc:1939] T 9260370cf8cb46a7b699581e4e8d46db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:32.965405 5383 raft_consensus.cc:3037] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:32.967078 6033 tablet_bootstrap.cc:654] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:32.975317 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01763d5cef124bad885eaaba0686787b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:32.978165 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 263) took 4869 ms. Trace:
W20260808 13:35:32.978621 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0b316390787f4170956f1c47a525d430 has no consensus state
I20260808 13:35:32.980187 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_43f5211ad48d49e982e3ac2625080359"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:32.982681 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_43f5211ad48d49e982e3ac2625080359 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:32.984799 7014 ts_tablet_manager.cc:1905] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:32.980881 6033 tablet_bootstrap.cc:492] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:32.988863 6033 ts_tablet_manager.cc:1397] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.048s user 0.012s sys 0.000s
I20260808 13:35:32.991878 6033 raft_consensus.cc:348] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.992527 6033 raft_consensus.cc:374] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:32.992849 6033 raft_consensus.cc:729] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:32.993458 6033 consensus_queue.cc:260] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:32.988076 7199 rpcz_store.cc:276] 0808 13:35:28.108433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.109643 (+ 1210us) service_pool.cc:227] Handling call
0808 13:35:32.810060 (+4700417us) ts_tablet_manager.cc:1145] Deleting tablet 9260370cf8cb46a7b699581e4e8d46db
0808 13:35:32.810109 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:32.855211 (+ 45102us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.952607 (+ 97396us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:32.978101 (+ 25494us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4691041}
I20260808 13:35:33.000077 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 9260370cf8cb46a7b699581e4e8d46db (table default.Table_4026b16c74ac4327ac4ba5f178bfcaf6 [id=b33430fc55db40dd8eb1c9e0396a093b]) successfully deleted
I20260808 13:35:33.001448 7199 tablet_replica.cc:330] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:33.001955 6046 tablet_bootstrap.cc:654] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:33.002784 7199 raft_consensus.cc:2229] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:33.043146 7199 raft_consensus.cc:2258] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:33.045400 7199 ts_tablet_manager.cc:1905] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:33.049770 6026 tablet_bootstrap.cc:492] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:33.050199 6026 ts_tablet_manager.cc:1397] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.236s user 0.009s sys 0.009s
I20260808 13:35:33.052516 6026 raft_consensus.cc:348] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.053189 6026 raft_consensus.cc:374] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:33.053431 6026 raft_consensus.cc:729] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:33.054112 6026 consensus_queue.cc:260] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:33.061537 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a73e98a8b20144508a5a8d451f4bb302 consensus state has no leader
I20260808 13:35:33.067757 6033 ts_tablet_manager.cc:1428] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.079s user 0.005s sys 0.002s
I20260808 13:35:33.068795 6033 tablet_bootstrap.cc:492] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:33.074918 4841 master_service.cc:946] ReplaceTablet: received request to replace tablet cb7f6487e28f4aaa94d448be71f8a3c2 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:33.078860 7199 ts_tablet_manager.cc:1918] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:33.079391 7199 log.cc:1196] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/a541ec35b1a44109866fabc953f14cc6
I20260808 13:35:33.080282 7199 ts_tablet_manager.cc:1939] T a541ec35b1a44109866fabc953f14cc6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:33.088780 5383 raft_consensus.cc:2454] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:33.088517 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_e4b2b77a6c9343b39de382dcd433c9e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:33.090096 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4b2b77a6c9343b39de382dcd433c9e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:33.099346 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 255) took 5078 ms. Trace:
I20260808 13:35:33.099697 7199 rpcz_store.cc:276] 0808 13:35:28.020973 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.186056 (+165083us) service_pool.cc:227] Handling call
0808 13:35:33.001263 (+4815207us) ts_tablet_manager.cc:1145] Deleting tablet a541ec35b1a44109866fabc953f14cc6
0808 13:35:33.001321 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.057255 (+ 55934us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.075167 (+ 17912us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.099271 (+ 24104us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":35782,"tablet-delete.queue_time_us":4813103}
I20260808 13:35:33.100966 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet a541ec35b1a44109866fabc953f14cc6 (table default.Table_0ba9f3241f494f148bd809e66279f480 [id=5b1fc9924efa44b8807de40565cdd419]) successfully deleted
I20260808 13:35:33.103966 6026 ts_tablet_manager.cc:1428] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.053s user 0.006s sys 0.000s
I20260808 13:35:33.104877 7387 raft_consensus.cc:482] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:33.105330 7387 raft_consensus.cc:504] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:33.107764 7387 leader_election.cc:288] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:33.113502 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5f0b33d3ad8942c0adb8ded9e15220c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:33.114338 7199 tablet_replica.cc:330] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
W20260808 13:35:33.115061 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f0b33d3ad8942c0adb8ded9e15220c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:33.115263 7199 raft_consensus.cc:2229] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:33.115839 7199 raft_consensus.cc:2258] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:33.116465 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf907a9f19f4d64b27be946011a8ddb" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:33.117141 5378 raft_consensus.cc:2454] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:33.118340 7199 ts_tablet_manager.cc:1905] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:33.119211 5771 leader_election.cc:302] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:33.124720 7387 raft_consensus.cc:2781] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:33.125272 7387 raft_consensus.cc:482] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:33.125736 7387 raft_consensus.cc:3037] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:33.129959 6033 tablet_bootstrap.cc:654] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:33.135354 6450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:35:33.141566 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf907a9f19f4d64b27be946011a8ddb" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:33.142254 5636 raft_consensus.cc:2454] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:33.145946 6026 tablet_bootstrap.cc:492] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:33.149847 7434 raft_consensus.cc:504] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:33.178177 5283 consensus_peers.cc:487] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 896389941179456d9d0ed3cd14726f11. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:33.012197 7013 ts_tablet_manager.cc:1918] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:33.188836 7013 log.cc:1196] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/6826e4e4db2043b3a4fe9e7a9359cab6
I20260808 13:35:33.195979 6026 tablet_bootstrap.cc:654] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:33.195935 7013 ts_tablet_manager.cc:1939] T 6826e4e4db2043b3a4fe9e7a9359cab6 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:33.200426 6450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:35:33.201033 6450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:35:33.211236 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 274) took 4731 ms. Trace:
I20260808 13:35:33.220695 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8514ca242bb2418d8ddbb88548b21832" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:33.221295 5882 raft_consensus.cc:3037] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:33.227624 7241 raft_consensus.cc:482] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:33.228103 7241 raft_consensus.cc:504] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.230360 7241 leader_election.cc:288] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:33.230844 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 1ba5fa465ec04bbb801a821a71a8f87d
I20260808 13:35:33.232547 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 3451d72548a140c68a6ccc2b566a09b1
I20260808 13:35:33.232820 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb8c7eeec164485a4e5bb80f9d5fd72" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:33.234568 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 3451d72548a140c68a6ccc2b566a09b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:32 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:33.235769 4833 catalog_manager.cc:6565] ReplaceTablet: tablet 1ba5fa465ec04bbb801a821a71a8f87d deleted and replaced by tablet 2483b3320e2e49db96d39254dd5755aa
I20260808 13:35:33.235697 5599 tablet_service.cc:1514] Processing DeleteTablet for tablet 3451d72548a140c68a6ccc2b566a09b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:32 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:33.236882 5599 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba5fa465ec04bbb801a821a71a8f87d with delete_type TABLET_DATA_DELETED (replaced by tablet 2483b3320e2e49db96d39254dd5755aa) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:33.236958 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_fb8b49faec0a429d95f595b94f65c8e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:33.237857 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_e37ea7df0fbb4dea827afe2023a0fac5" } new_table_name: "default.Table_5b856c2026594d708a6d8e38edd5d85f" modify_external_catalogs: true
I20260808 13:35:33.238440 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba5fa465ec04bbb801a821a71a8f87d with delete_type TABLET_DATA_DELETED (replaced by tablet 2483b3320e2e49db96d39254dd5755aa) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:33.240757 5772 leader_election.cc:341] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:33.249208 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb8b49faec0a429d95f595b94f65c8e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:33.251236 5882 raft_consensus.cc:2454] T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:33.251924 7434 leader_election.cc:288] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:33.204142 6046 tablet_bootstrap.cc:492] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:33.255856 5550 leader_election.cc:302] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:33.257820 7470 raft_consensus.cc:2781] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
W20260808 13:35:33.263018 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d0cc688386f3413ea939dda3a75d6541 consensus state has no leader
I20260808 13:35:33.264235 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_90f7539628894de2821bdf85dfdf4498"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:33.266189 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90f7539628894de2821bdf85dfdf4498 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:33.272502 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:33.270171 7470 raft_consensus.cc:686] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:33.274024 7470 consensus_queue.cc:237] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.279007 7387 raft_consensus.cc:504] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:33.284965 5599 tablet_service.cc:1467] Processing CreateTablet for tablet ad151fca07264d9bae6e07ca781c8e04 (DEFAULT_TABLE table=default.Table_ddfcf011ede6442fbc7a45c799fc2cc6 [id=f3bc7b846abd4480b1c1c927c8ad0953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.286072 5599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad151fca07264d9bae6e07ca781c8e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.285887 5841 tablet_service.cc:1467] Processing CreateTablet for tablet ad151fca07264d9bae6e07ca781c8e04 (DEFAULT_TABLE table=default.Table_ddfcf011ede6442fbc7a45c799fc2cc6 [id=f3bc7b846abd4480b1c1c927c8ad0953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.287254 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad151fca07264d9bae6e07ca781c8e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.294253 5600 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0018c5333b4fde8ad4ce874660eb97 (DEFAULT_TABLE table=default.Table_1e7d0ae2ebb1437c83136cc4e91e8a23 [id=86e7a76979764ac09c964217ebe7fae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.295509 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0018c5333b4fde8ad4ce874660eb97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.295357 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0018c5333b4fde8ad4ce874660eb97 (DEFAULT_TABLE table=default.Table_1e7d0ae2ebb1437c83136cc4e91e8a23 [id=86e7a76979764ac09c964217ebe7fae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.296551 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0018c5333b4fde8ad4ce874660eb97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.298964 4837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 61dfd47db0a74ed8b7228b559dd68459
I20260808 13:35:33.298827 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 01982f911d394aee8f4a3cd3b7110100 (DEFAULT_TABLE table=default.Table_83babd1cd9304044b4bdb5cf0ee909a4 [id=b1e1dc25e25a4075ae747315c8130723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.299839 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01982f911d394aee8f4a3cd3b7110100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.302152 4839 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:33.303027 4839 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_953730643273433c8353604101a4d237 [id=d9baa62ea5c249a898586a45f8b4f372]). Expected version 2 got 1
I20260808 13:35:33.305094 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 61dfd47db0a74ed8b7228b559dd68459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:32 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:33.306238 5613 tablet_service.cc:1467] Processing CreateTablet for tablet b8373a633a07432b9efdf49e932d10f1 (DEFAULT_TABLE table=default.Table_1ab5d21bca4a47978a8f5483f2ba10b2 [id=6d0c2c16340f4cd8805c7692e6275a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.307286 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8373a633a07432b9efdf49e932d10f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.308683 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 61dfd47db0a74ed8b7228b559dd68459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:32 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:33.309928 5832 tablet_service.cc:1467] Processing CreateTablet for tablet 01982f911d394aee8f4a3cd3b7110100 (DEFAULT_TABLE table=default.Table_83babd1cd9304044b4bdb5cf0ee909a4 [id=b1e1dc25e25a4075ae747315c8130723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.311611 5832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01982f911d394aee8f4a3cd3b7110100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.317216 6033 tablet_bootstrap.cc:492] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:33.317652 6033 ts_tablet_manager.cc:1397] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.249s user 0.008s sys 0.001s
I20260808 13:35:33.320935 7014 ts_tablet_manager.cc:1918] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:33.321411 7014 log.cc:1196] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/127e8d0323954ef788018bcb91582951
I20260808 13:35:33.322301 7014 ts_tablet_manager.cc:1939] T 127e8d0323954ef788018bcb91582951 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:33.325024 5606 tablet_service.cc:1467] Processing CreateTablet for tablet ab86016e556c4cdcafeaeda32f56a836 (DEFAULT_TABLE table=default.Table_3961834935a84f0a9646f77db54765ee [id=5115ecc58364490c953100f78c49facf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.326083 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab86016e556c4cdcafeaeda32f56a836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.329033 5607 tablet_service.cc:1467] Processing CreateTablet for tablet 4747a38c520d40848f4d3c261499aa88 (DEFAULT_TABLE table=default.Table_a384cb98038948a2aa2ea653c22ac2eb [id=334643ef0e864267b0f17674d4f1c06c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.329807 4839 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:33.330083 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4747a38c520d40848f4d3c261499aa88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.331599 4839 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:33.339385 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 266) took 4818 ms. Trace:
I20260808 13:35:33.339751 7014 rpcz_store.cc:276] 0808 13:35:28.521401 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.521494 (+ 93us) service_pool.cc:227] Handling call
0808 13:35:32.936660 (+4415166us) ts_tablet_manager.cc:1145] Deleting tablet 127e8d0323954ef788018bcb91582951
0808 13:35:32.936700 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.102158 (+165458us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.192348 (+ 90190us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.339319 (+146971us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4391727}
I20260808 13:35:33.340502 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 8d06a8cdfabd4adaa4fef064ee6da7d3 (DEFAULT_TABLE table=default.Table_5398341b051f491ca73c85f9c8332eb9 [id=9dbadd28dd50468ba9a99548e008600f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.341142 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 267) took 4476 ms. Trace:
I20260808 13:35:33.341429 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d06a8cdfabd4adaa4fef064ee6da7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.342924 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 127e8d0323954ef788018bcb91582951 (table default.Table_23396b27c08444c7981e55a66139f6b6 [id=2004b79d44ab440ba8d0004db91b6bce]) successfully deleted
I20260808 13:35:33.348620 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 0b316390787f4170956f1c47a525d430 (DEFAULT_TABLE table=default.Table_f3144416bf1140aaaa78d9ec41319c6b [id=abda1a6df047407ca80dadf6d4ccb804]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.349807 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b316390787f4170956f1c47a525d430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.350477 5834 tablet_service.cc:1467] Processing CreateTablet for tablet b8373a633a07432b9efdf49e932d10f1 (DEFAULT_TABLE table=default.Table_1ab5d21bca4a47978a8f5483f2ba10b2 [id=6d0c2c16340f4cd8805c7692e6275a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.351655 5834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8373a633a07432b9efdf49e932d10f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.359417 5615 tablet_service.cc:1467] Processing CreateTablet for tablet 0329c348c88a4ddda0f2b90be7a4af76 (DEFAULT_TABLE table=default.Table_4ef3501cc89b4779b73a48142fd01d36 [id=b449f7fe2512431790a1a39a4dde3570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.360497 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0329c348c88a4ddda0f2b90be7a4af76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.361941 5616 tablet_service.cc:1467] Processing CreateTablet for tablet ce39289278cb44cc86d41c3abb516a1d (DEFAULT_TABLE table=default.Table_9758d29f1a0b4ea5bceef3d5ce652ca1 [id=a3d0de3d513c43f29bea23a52883916a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.362946 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce39289278cb44cc86d41c3abb516a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.365823 5836 tablet_service.cc:1467] Processing CreateTablet for tablet ab86016e556c4cdcafeaeda32f56a836 (DEFAULT_TABLE table=default.Table_3961834935a84f0a9646f77db54765ee [id=5115ecc58364490c953100f78c49facf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.367064 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab86016e556c4cdcafeaeda32f56a836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.367784 5609 tablet_service.cc:1467] Processing CreateTablet for tablet 315c8b665cc9490da0b10623b9963154 (DEFAULT_TABLE table=default.Table_94d7e54302ae4463a43a530d34d4f264 [id=062b1b88fea4482fa36b7456f6534f4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.368851 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 315c8b665cc9490da0b10623b9963154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.372654 5845 tablet_service.cc:1467] Processing CreateTablet for tablet 4747a38c520d40848f4d3c261499aa88 (DEFAULT_TABLE table=default.Table_a384cb98038948a2aa2ea653c22ac2eb [id=334643ef0e864267b0f17674d4f1c06c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.373770 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4747a38c520d40848f4d3c261499aa88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.379817 6033 raft_consensus.cc:348] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.380522 6033 raft_consensus.cc:374] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:33.380951 6033 raft_consensus.cc:729] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:33.381675 6033 consensus_queue.cc:260] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.386919 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2335c2182d4c2abaa254f4554fd473 (DEFAULT_TABLE table=default.Table_e916116a3ec24469b4670c87eeaac98e [id=f348502b786943fdb6bcb104e73355f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.387960 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b2335c2182d4c2abaa254f4554fd473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.388558 6026 tablet_bootstrap.cc:492] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:33.389099 6026 ts_tablet_manager.cc:1397] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.243s user 0.009s sys 0.001s
I20260808 13:35:33.396184 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 70ec24d1bd3a42b0af377a6461f9264d (DEFAULT_TABLE table=default.Table_7e9aac444b5a43f29dc7608e6ef379f3 [id=6a2ea5033865443d85ac6e6429054471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.397274 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ec24d1bd3a42b0af377a6461f9264d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.397212 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_b84003117fd04b17930f7a3ba4e2d05f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:33.403461 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b84003117fd04b17930f7a3ba4e2d05f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:33.403999 5598 tablet_service.cc:1467] Processing CreateTablet for tablet e5fb1d373c4d486ca3815023897721a2 (DEFAULT_TABLE table=default.Table_2a111b21bed74a8bb3c9822dff0c2cc6 [id=6a8b8e54b4b24c0c84ac855dea6afd49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.405113 5598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fb1d373c4d486ca3815023897721a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.407785 4841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet cb7f6487e28f4aaa94d448be71f8a3c2
I20260808 13:35:33.409111 4841 catalog_manager.cc:6565] ReplaceTablet: tablet cb7f6487e28f4aaa94d448be71f8a3c2 deleted and replaced by tablet d0767140736242f18ef4689b112deefb
I20260808 13:35:33.412355 5840 tablet_service.cc:1467] Processing CreateTablet for tablet 8d06a8cdfabd4adaa4fef064ee6da7d3 (DEFAULT_TABLE table=default.Table_5398341b051f491ca73c85f9c8332eb9 [id=9dbadd28dd50468ba9a99548e008600f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.412757 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 0b316390787f4170956f1c47a525d430 (DEFAULT_TABLE table=default.Table_f3144416bf1140aaaa78d9ec41319c6b [id=abda1a6df047407ca80dadf6d4ccb804]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.413475 5840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d06a8cdfabd4adaa4fef064ee6da7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.414247 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b316390787f4170956f1c47a525d430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.417958 7199 ts_tablet_manager.cc:1918] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:33.418521 7199 log.cc:1196] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/4625b632d67244fc86e48d739dde707e
I20260808 13:35:33.419404 7199 ts_tablet_manager.cc:1939] T 4625b632d67244fc86e48d739dde707e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:33.422577 7241 consensus_queue.cc:1035] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:33.427181 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet cb7f6487e28f4aaa94d448be71f8a3c2 with delete_type TABLET_DATA_DELETED (replaced by tablet d0767140736242f18ef4689b112deefb) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:33.430215 5831 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2335c2182d4c2abaa254f4554fd473 (DEFAULT_TABLE table=default.Table_e916116a3ec24469b4670c87eeaac98e [id=f348502b786943fdb6bcb104e73355f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.424220 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 0329c348c88a4ddda0f2b90be7a4af76 (DEFAULT_TABLE table=default.Table_4ef3501cc89b4779b73a48142fd01d36 [id=b449f7fe2512431790a1a39a4dde3570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.430855 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 107309c7ada246b6a15a784d7a8c399d (DEFAULT_TABLE table=default.Table_025eb6ef4e024bec917c3272ba1b8c35 [id=7dda6de1004344a987b5ede426b5a758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.431995 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107309c7ada246b6a15a784d7a8c399d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.432528 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 70ec24d1bd3a42b0af377a6461f9264d (DEFAULT_TABLE table=default.Table_7e9aac444b5a43f29dc7608e6ef379f3 [id=6a2ea5033865443d85ac6e6429054471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.434491 5829 tablet_service.cc:1467] Processing CreateTablet for tablet e5fb1d373c4d486ca3815023897721a2 (DEFAULT_TABLE table=default.Table_2a111b21bed74a8bb3c9822dff0c2cc6 [id=6a8b8e54b4b24c0c84ac855dea6afd49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.435981 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet cb7f6487e28f4aaa94d448be71f8a3c2 with delete_type TABLET_DATA_DELETED (replaced by tablet d0767140736242f18ef4689b112deefb) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:33.436645 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_ce81522648e14e5daa8c8fb7d55ef7e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:33.438220 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce81522648e14e5daa8c8fb7d55ef7e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:33.444962 5592 tablet_service.cc:1467] Processing CreateTablet for tablet 4561c9af06e14ed8a2ec4c7285e2ad99 (DEFAULT_TABLE table=default.Table_248e910ea5a34c55978753b74fe71832 [id=f58086e90fc34ca3a5d31417a82dad6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.446202 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4561c9af06e14ed8a2ec4c7285e2ad99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.460355 6026 raft_consensus.cc:348] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.461412 6026 raft_consensus.cc:374] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:33.461750 6026 raft_consensus.cc:729] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:33.462234 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:33.341454 7014 rpcz_store.cc:276] 0808 13:35:28.864868 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.865142 (+ 274us) service_pool.cc:227] Handling call
0808 13:35:33.340720 (+4475578us) ts_tablet_manager.cc:1145] Deleting tablet 9f53e5b5c4bd4a9385ad198b283c6e79
0808 13:35:33.340792 (+ 72us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.341089 (+ 297us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4475027}
I20260808 13:35:33.463529 7014 tablet_replica.cc:330] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:33.464298 7014 raft_consensus.cc:2229] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:33.464777 7014 raft_consensus.cc:2258] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:33.467880 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 9f53e5b5c4bd4a9385ad198b283c6e79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9f53e5b5c4bd4a9385ad198b283c6e79 already in progress: creating tablet
I20260808 13:35:33.384048 6033 ts_tablet_manager.cc:1428] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.066s user 0.007s sys 0.000s
I20260808 13:35:33.479910 5827 tablet_service.cc:1467] Processing CreateTablet for tablet 107309c7ada246b6a15a784d7a8c399d (DEFAULT_TABLE table=default.Table_025eb6ef4e024bec917c3272ba1b8c35 [id=7dda6de1004344a987b5ede426b5a758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.487694 7561 raft_consensus.cc:482] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:33.488287 5826 tablet_service.cc:1467] Processing CreateTablet for tablet 4561c9af06e14ed8a2ec4c7285e2ad99 (DEFAULT_TABLE table=default.Table_248e910ea5a34c55978753b74fe71832 [id=f58086e90fc34ca3a5d31417a82dad6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.488358 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:33.490178 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:33.488950 7561 raft_consensus.cc:504] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.507048 6026 consensus_queue.cc:260] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.538162 6046 ts_tablet_manager.cc:1397] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.585s user 0.005s sys 0.008s
I20260808 13:35:33.548465 6026 ts_tablet_manager.cc:1428] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.159s user 0.009s sys 0.000s
I20260808 13:35:33.426062 5843 tablet_service.cc:1467] Processing CreateTablet for tablet ce39289278cb44cc86d41c3abb516a1d (DEFAULT_TABLE table=default.Table_9758d29f1a0b4ea5bceef3d5ce652ca1 [id=a3d0de3d513c43f29bea23a52883916a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.427829 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 315c8b665cc9490da0b10623b9963154 (DEFAULT_TABLE table=default.Table_94d7e54302ae4463a43a530d34d4f264 [id=062b1b88fea4482fa36b7456f6534f4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.497206 5831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b2335c2182d4c2abaa254f4554fd473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.513753 6033 tablet_bootstrap.cc:492] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:33.556499 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ec24d1bd3a42b0af377a6461f9264d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.559989 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 315c8b665cc9490da0b10623b9963154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.560981 5827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107309c7ada246b6a15a784d7a8c399d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.562963 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce39289278cb44cc86d41c3abb516a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.566087 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0329c348c88a4ddda0f2b90be7a4af76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.574647 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fb1d373c4d486ca3815023897721a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.578914 5826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4561c9af06e14ed8a2ec4c7285e2ad99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.257090 7013 rpcz_store.cc:276] 0808 13:35:28.479784 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.479880 (+ 96us) service_pool.cc:227] Handling call
0808 13:35:32.530691 (+4050811us) ts_tablet_manager.cc:1145] Deleting tablet 6826e4e4db2043b3a4fe9e7a9359cab6
0808 13:35:32.530735 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:32.627421 (+ 96686us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.011792 (+384371us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.211146 (+199354us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4050053}
I20260808 13:35:33.424854 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 265) took 5238 ms. Trace:
I20260808 13:35:33.582798 7199 rpcz_store.cc:276] 0808 13:35:28.186582 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.186838 (+ 256us) service_pool.cc:227] Handling call
0808 13:35:33.101608 (+4914770us) ts_tablet_manager.cc:1145] Deleting tablet 4625b632d67244fc86e48d739dde707e
0808 13:35:33.114185 (+ 12577us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.164304 (+ 50119us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.417732 (+253428us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.424792 (+ 7060us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4913384}
I20260808 13:35:33.584291 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 275) took 5102 ms. Trace:
I20260808 13:35:33.584944 7199 tablet_replica.cc:330] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:33.584712 7013 rpcz_store.cc:276] 0808 13:35:28.482140 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.482252 (+ 112us) service_pool.cc:227] Handling call
0808 13:35:33.583346 (+5101094us) ts_tablet_manager.cc:1145] Deleting tablet 2173e97590be43f9a24f55689d796269
0808 13:35:33.583609 (+ 263us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.584145 (+ 536us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5099944}
I20260808 13:35:33.585770 7199 raft_consensus.cc:2229] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:33.586242 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 278) took 4754 ms. Trace:
I20260808 13:35:33.586521 7199 raft_consensus.cc:2258] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:33.586589 7013 rpcz_store.cc:276] 0808 13:35:28.831521 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.834126 (+ 2605us) service_pool.cc:227] Handling call
0808 13:35:33.585716 (+4751590us) ts_tablet_manager.cc:1145] Deleting tablet 9f53e5b5c4bd4a9385ad198b283c6e79
0808 13:35:33.585889 (+ 173us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.586195 (+ 306us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4750448}
I20260808 13:35:33.589006 7199 ts_tablet_manager.cc:1905] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:33.598908 7562 raft_consensus.cc:482] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:33.600901 7562 raft_consensus.cc:504] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.623178 7014 ts_tablet_manager.cc:1905] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:33.596577 7561 leader_election.cc:288] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:33.599946 6033 tablet_bootstrap.cc:654] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:33.639495 7387 leader_election.cc:288] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:33.491034 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_93dd48e3ed114ba0a47115dac0dfcd7f" } modify_external_catalogs: true
I20260808 13:35:33.666168 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_93dd48e3ed114ba0a47115dac0dfcd7f" } modify_external_catalogs: true
I20260808 13:35:33.584260 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 4625b632d67244fc86e48d739dde707e (table default.Table_1665f65d290446e79ac5f6550ec733b7 [id=2f47077a0fd6488ebc33ab8794c0a5b6]) successfully deleted
W20260808 13:35:33.670883 4819 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 86667.7us
I20260808 13:35:33.653107 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b2521a9f129840958fb2b9097b7cf37b" } new_table_name: "default.Table_f6d782e61daf4dd5be5685aea75e9fd2" modify_external_catalogs: true
I20260808 13:35:33.678783 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:33.686276 6033 tablet_bootstrap.cc:492] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:33.686830 6033 ts_tablet_manager.cc:1397] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.182s user 0.010s sys 0.001s
I20260808 13:35:33.689352 6033 raft_consensus.cc:348] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.689973 6033 raft_consensus.cc:374] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:33.690213 6033 raft_consensus.cc:729] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:33.690788 6033 consensus_queue.cc:260] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.715202 7562 leader_election.cc:288] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:33.628026 4784 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 564952us
I20260808 13:35:33.726454 7387 raft_consensus.cc:482] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:33.726966 7387 raft_consensus.cc:504] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.729254 7387 leader_election.cc:288] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:33.588784 6046 raft_consensus.cc:348] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.732611 6046 raft_consensus.cc:374] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:33.732921 7013 tablet_replica.cc:330] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:33.732940 6046 raft_consensus.cc:729] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:33.733502 6033 ts_tablet_manager.cc:1428] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.046s user 0.006s sys 0.000s
I20260808 13:35:33.733937 7013 raft_consensus.cc:2229] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:33.733695 6046 consensus_queue.cc:260] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.734548 7013 raft_consensus.cc:2258] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:33.736042 6046 ts_tablet_manager.cc:1428] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.197s user 0.006s sys 0.000s
I20260808 13:35:33.736994 6046 tablet_bootstrap.cc:492] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:33.618276 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb8c7eeec164485a4e5bb80f9d5fd72" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:33.618556 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf907a9f19f4d64b27be946011a8ddb" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:33.683671 7575 consensus_queue.cc:1035] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:33.742988 5379 raft_consensus.cc:3037] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:33.747366 7014 ts_tablet_manager.cc:1918] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:33.747871 7014 log.cc:1196] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/ddfd9e3f002445738df24ebb0a333440
I20260808 13:35:33.748739 7014 ts_tablet_manager.cc:1939] T ddfd9e3f002445738df24ebb0a333440 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:33.753021 7199 ts_tablet_manager.cc:1918] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:33.754139 7199 log.cc:1196] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/5d388589d386408bb10ca030b87af787
I20260808 13:35:33.754369 5379 raft_consensus.cc:2454] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:33.755221 7199 ts_tablet_manager.cc:1939] T 5d388589d386408bb10ca030b87af787 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
W20260808 13:35:33.769840 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 350482us
I20260808 13:35:33.770749 6046 tablet_bootstrap.cc:654] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:33.779683 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264ee45f8dfa4a05b0e5e812ce1b622c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:33.780719 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9268f5f6dee4112932e106010a3c2be" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:33.788674 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf907a9f19f4d64b27be946011a8ddb" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:33.788955 6026 tablet_bootstrap.cc:492] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:33.789155 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 268) took 4619 ms. Trace:
I20260808 13:35:33.789270 5630 raft_consensus.cc:3037] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:33.789426 7014 rpcz_store.cc:276] 0808 13:35:29.170095 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:29.170203 (+ 108us) service_pool.cc:227] Handling call
0808 13:35:33.463360 (+4293157us) ts_tablet_manager.cc:1145] Deleting tablet ddfd9e3f002445738df24ebb0a333440
0808 13:35:33.463413 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.630281 (+166868us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.747061 (+116780us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.789095 (+ 42034us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4292563}
I20260808 13:35:33.790120 7552 raft_consensus.cc:482] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:33.790731 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 274) took 4140 ms. Trace:
I20260808 13:35:33.791023 7014 rpcz_store.cc:276] 0808 13:35:29.650832 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:29.663950 (+ 13118us) service_pool.cc:227] Handling call
0808 13:35:33.790385 (+4126435us) ts_tablet_manager.cc:1145] Deleting tablet 0756b5f24ed94a9fbe2a43efc7b8d40d
0808 13:35:33.790428 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.790675 (+ 247us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4125907}
I20260808 13:35:33.797636 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
W20260808 13:35:33.704059 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 52deb2e968c04b99a436e04ae78d9969 consensus state has no leader
I20260808 13:35:33.798908 4835 catalog_manager.cc:5495] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:33.707378 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 293e9540498e420d8ec873472d0d29fc
I20260808 13:35:33.783643 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 6826e4e4db2043b3a4fe9e7a9359cab6 (table default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]) successfully deleted
I20260808 13:35:33.803411 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9423b380a62c467895e7fdff14756586" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:33.804018 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 2173e97590be43f9a24f55689d796269 because the tablet was not found. No further retry: Not found: Tablet not found: 2173e97590be43f9a24f55689d796269
I20260808 13:35:33.804143 5883 raft_consensus.cc:2454] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
W20260808 13:35:33.804572 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 9f53e5b5c4bd4a9385ad198b283c6e79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9f53e5b5c4bd4a9385ad198b283c6e79 already in progress: creating tablet
I20260808 13:35:33.791908 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]) successfully deleted
W20260808 13:35:33.805490 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0756b5f24ed94a9fbe2a43efc7b8d40d already in progress: creating tablet
I20260808 13:35:33.806066 7013 ts_tablet_manager.cc:1905] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:33.808532 5550 leader_election.cc:302] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:33.809443 7560 raft_consensus.cc:482] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:33.809836 7560 raft_consensus.cc:504] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.810899 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 293e9540498e420d8ec873472d0d29fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:33 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:33.818035 7560 leader_election.cc:288] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:33.821094 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_90f7539628894de2821bdf85dfdf4498" } modify_external_catalogs: true
I20260808 13:35:33.821751 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_90f7539628894de2821bdf85dfdf4498" } modify_external_catalogs: true
I20260808 13:35:33.823256 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba5fa465ec04bbb801a821a71a8f87d with delete_type TABLET_DATA_DELETED (replaced by tablet 2483b3320e2e49db96d39254dd5755aa) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:33.823570 5355 tablet_service.cc:1514] Processing DeleteTablet for tablet 3451d72548a140c68a6ccc2b566a09b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:32 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:33.824716 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0018c5333b4fde8ad4ce874660eb97 (DEFAULT_TABLE table=default.Table_1e7d0ae2ebb1437c83136cc4e91e8a23 [id=86e7a76979764ac09c964217ebe7fae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.824913 5354 tablet_service.cc:1467] Processing CreateTablet for tablet ad151fca07264d9bae6e07ca781c8e04 (DEFAULT_TABLE table=default.Table_ddfcf011ede6442fbc7a45c799fc2cc6 [id=f3bc7b846abd4480b1c1c927c8ad0953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.825847 5345 tablet_service.cc:1514] Processing DeleteTablet for tablet 61dfd47db0a74ed8b7228b559dd68459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:32 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:33.826226 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_2269d7b97e6a488b9a86e8240b3b197e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:33.827211 5336 tablet_service.cc:1467] Processing CreateTablet for tablet b8373a633a07432b9efdf49e932d10f1 (DEFAULT_TABLE table=default.Table_1ab5d21bca4a47978a8f5483f2ba10b2 [id=6d0c2c16340f4cd8805c7692e6275a66]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:33.827802 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2269d7b97e6a488b9a86e8240b3b197e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:33.828895 5355 tablet_service.cc:1467] Processing CreateTablet for tablet 01982f911d394aee8f4a3cd3b7110100 (DEFAULT_TABLE table=default.Table_83babd1cd9304044b4bdb5cf0ee909a4 [id=b1e1dc25e25a4075ae747315c8130723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.833401 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 293e9540498e420d8ec873472d0d29fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:33 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:33.835539 7560 raft_consensus.cc:482] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:33.835973 7560 raft_consensus.cc:504] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:33.836946 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0018c5333b4fde8ad4ce874660eb97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.837781 7560 leader_election.cc:288] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:33.839545 6033 tablet_bootstrap.cc:492] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:33.840560 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad151fca07264d9bae6e07ca781c8e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.844516 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 271) took 5467 ms. Trace:
I20260808 13:35:33.844718 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8373a633a07432b9efdf49e932d10f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.844796 7199 rpcz_store.cc:276] 0808 13:35:28.377287 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.390684 (+ 13397us) service_pool.cc:227] Handling call
0808 13:35:33.584783 (+5194099us) ts_tablet_manager.cc:1145] Deleting tablet 5d388589d386408bb10ca030b87af787
0808 13:35:33.584841 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.699337 (+114496us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.752736 (+ 53399us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:33.803687 (+ 50951us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b5800001898. stack: 00007f19df247033 00007f19df246f30 00007f19dedc8534 00000000004db2b4 00000000004db249 00007f19df0bdf11 00007f19df241851 00007f19df241e82 00007f19e81018b9 00007f19e71355d8 00007f19e71f9e42 00007f19e7229e10 00007f19e7210e0d 00007f19e7210db9 00007f19e7210d49 00007f19e7210d11
0808 13:35:33.844458 (+ 40771us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":92190464,"tablet-delete.queue_time_us":5176149}
I20260808 13:35:33.845834 7199 tablet_replica.cc:330] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:33.847842 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 5d388589d386408bb10ca030b87af787 (table default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88]) successfully deleted
I20260808 13:35:33.790524 7552 raft_consensus.cc:504] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.796044 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9268f5f6dee4112932e106010a3c2be" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:33.846525 7199 raft_consensus.cc:2229] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:33.850955 7199 raft_consensus.cc:2258] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:33.852982 7199 ts_tablet_manager.cc:1905] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:33.854902 5355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01982f911d394aee8f4a3cd3b7110100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.856055 4833 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet dc8069561e074be0b325a88c1129bd5a
I20260808 13:35:33.857856 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_7e0bc5965bd54d788347d0d41c53208f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:33.859697 6033 tablet_bootstrap.cc:654] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:33.861251 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e0bc5965bd54d788347d0d41c53208f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:33.888092 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264ee45f8dfa4a05b0e5e812ce1b622c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:33.892079 6046 tablet_bootstrap.cc:492] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:33.892539 6046 ts_tablet_manager.cc:1397] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.156s user 0.005s sys 0.005s
I20260808 13:35:33.894068 5345 tablet_service.cc:1467] Processing CreateTablet for tablet ab86016e556c4cdcafeaeda32f56a836 (DEFAULT_TABLE table=default.Table_3961834935a84f0a9646f77db54765ee [id=5115ecc58364490c953100f78c49facf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.895264 5345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab86016e556c4cdcafeaeda32f56a836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.898392 5882 raft_consensus.cc:3037] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:33.903209 6046 raft_consensus.cc:348] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.903940 6046 raft_consensus.cc:374] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:33.904212 6046 raft_consensus.cc:729] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:33.905094 7014 tablet_replica.cc:330] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:33.904914 6046 consensus_queue.cc:260] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.905962 7014 raft_consensus.cc:2229] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:33.906502 7014 raft_consensus.cc:2258] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:33.908375 6026 tablet_bootstrap.cc:654] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:33.910801 6046 ts_tablet_manager.cc:1428] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.018s user 0.007s sys 0.001s
I20260808 13:35:33.911453 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 8d06a8cdfabd4adaa4fef064ee6da7d3 (DEFAULT_TABLE table=default.Table_5398341b051f491ca73c85f9c8332eb9 [id=9dbadd28dd50468ba9a99548e008600f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.911550 5335 tablet_service.cc:1467] Processing CreateTablet for tablet 4747a38c520d40848f4d3c261499aa88 (DEFAULT_TABLE table=default.Table_a384cb98038948a2aa2ea653c22ac2eb [id=334643ef0e864267b0f17674d4f1c06c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.912919 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d06a8cdfabd4adaa4fef064ee6da7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.913908 5335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4747a38c520d40848f4d3c261499aa88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.888969 7434 raft_consensus.cc:2781] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:33.920480 7014 ts_tablet_manager.cc:1905] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:33.920879 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 0b316390787f4170956f1c47a525d430 (DEFAULT_TABLE table=default.Table_f3144416bf1140aaaa78d9ec41319c6b [id=abda1a6df047407ca80dadf6d4ccb804]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.922154 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b316390787f4170956f1c47a525d430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.916385 7434 raft_consensus.cc:482] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:33.923255 7434 raft_consensus.cc:3037] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:33.923592 6046 tablet_bootstrap.cc:492] T 8514ca242bb2418d8ddbb88548b21832 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:33.925240 5334 tablet_service.cc:1467] Processing CreateTablet for tablet 0329c348c88a4ddda0f2b90be7a4af76 (DEFAULT_TABLE table=default.Table_4ef3501cc89b4779b73a48142fd01d36 [id=b449f7fe2512431790a1a39a4dde3570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.931362 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0329c348c88a4ddda0f2b90be7a4af76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.944922 6046 tablet_bootstrap.cc:654] T 8514ca242bb2418d8ddbb88548b21832 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:33.948758 7552 leader_election.cc:288] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:33.908394 7470 raft_consensus.cc:482] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:33.963317 5337 tablet_service.cc:1467] Processing CreateTablet for tablet ce39289278cb44cc86d41c3abb516a1d (DEFAULT_TABLE table=default.Table_9758d29f1a0b4ea5bceef3d5ce652ca1 [id=a3d0de3d513c43f29bea23a52883916a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.957558 7470 raft_consensus.cc:504] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:33.968580 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce39289278cb44cc86d41c3abb516a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.981058 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 315c8b665cc9490da0b10623b9963154 (DEFAULT_TABLE table=default.Table_94d7e54302ae4463a43a530d34d4f264 [id=062b1b88fea4482fa36b7456f6534f4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:33.983124 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 315c8b665cc9490da0b10623b9963154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:33.988387 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba098c946794607b2a15bc64098d918" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:33.989650 5636 raft_consensus.cc:2454] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:33.994560 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ba43d1479a4f9e9d08bdfa29e7157b" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:33.994771 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0cc688386f3413ea939dda3a75d6541" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:33.998149 7470 leader_election.cc:288] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:33.966281 5630 raft_consensus.cc:2454] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:34.004209 5338 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2335c2182d4c2abaa254f4554fd473 (DEFAULT_TABLE table=default.Table_e916116a3ec24469b4670c87eeaac98e [id=f348502b786943fdb6bcb104e73355f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.005508 5338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b2335c2182d4c2abaa254f4554fd473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.006323 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1847d0fecb843cdbc833ad674a503a4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:34.006984 5883 raft_consensus.cc:2454] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:34.009119 6033 tablet_bootstrap.cc:492] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:34.009538 6033 ts_tablet_manager.cc:1397] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.170s user 0.007s sys 0.006s
W20260808 13:35:34.010457 5550 leader_election.cc:341] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.011059 5550 leader_election.cc:302] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:35:34.019632 5598 consensus_meta.cc:287] T e5fb1d373c4d486ca3815023897721a2 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.605s user 0.007s sys 0.000s
I20260808 13:35:34.013976 7562 raft_consensus.cc:2781] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:34.021962 7562 raft_consensus.cc:482] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260808 13:35:34.028826 5772 consensus_peers.cc:487] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ddfd9e3f002445738df24ebb0a333440. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:34.029814 5772 leader_election.cc:341] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.035024 7566 consensus_queue.cc:1035] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:35:34.029965 5595 consensus_meta.cc:287] T 107309c7ada246b6a15a784d7a8c399d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.513s user 0.008s sys 0.000s
I20260808 13:35:34.033577 7562 raft_consensus.cc:3037] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:34.039762 5600 consensus_meta.cc:287] T 0d0018c5333b4fde8ad4ce874660eb97 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent flushing consensus metadata: real 0.544s user 0.007s sys 0.001s
I20260808 13:35:34.051800 6033 raft_consensus.cc:348] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:34.052611 5285 consensus_peers.cc:487] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e00f2e0b98af431d99894e8778257bc4. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:34.054045 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 269978us
I20260808 13:35:34.055785 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 107309c7ada246b6a15a784d7a8c399d (DEFAULT_TABLE table=default.Table_025eb6ef4e024bec917c3272ba1b8c35 [id=7dda6de1004344a987b5ede426b5a758]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:34.056556 5771 consensus_peers.cc:487] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:34.057718 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet cb7f6487e28f4aaa94d448be71f8a3c2 with delete_type TABLET_DATA_DELETED (replaced by tablet d0767140736242f18ef4689b112deefb) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:34.059319 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 4561c9af06e14ed8a2ec4c7285e2ad99 (DEFAULT_TABLE table=default.Table_248e910ea5a34c55978753b74fe71832 [id=f58086e90fc34ca3a5d31417a82dad6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.060959 5346 tablet_service.cc:1514] Processing DeleteTablet for tablet 293e9540498e420d8ec873472d0d29fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:33 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:34.062367 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8514ca242bb2418d8ddbb88548b21832" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:34.063923 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9423b380a62c467895e7fdff14756586" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:34.064457 5383 raft_consensus.cc:2454] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
W20260808 13:35:34.065807 5771 leader_election.cc:341] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.066162 5771 leader_election.cc:302] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:34.069929 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0cc688386f3413ea939dda3a75d6541" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:34.071235 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6fc58ab15c1b44c69500f3ad87062bcd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:34.073156 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6fc58ab15c1b44c69500f3ad87062bcd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:34.074635 5772 consensus_peers.cc:487] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 127e8d0323954ef788018bcb91582951. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:34.075026 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 44535.3us
W20260808 13:35:34.075562 5772 leader_election.cc:341] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.075675 7434 raft_consensus.cc:504] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.075919 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_51d39ab358f3470084d7cd6e270eee0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:34.077409 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51d39ab358f3470084d7cd6e270eee0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:34.052546 6033 raft_consensus.cc:374] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:34.078933 6033 raft_consensus.cc:729] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:34.079085 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9423b380a62c467895e7fdff14756586" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:34.079658 5883 raft_consensus.cc:3037] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:34.079779 6033 consensus_queue.cc:260] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.080693 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107309c7ada246b6a15a784d7a8c399d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.081990 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4561c9af06e14ed8a2ec4c7285e2ad99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.085933 7241 raft_consensus.cc:2726] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:34.081530 7590 consensus_queue.cc:1035] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260808 13:35:34.086308 5883 raft_consensus.cc:2454] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:34.087441 5550 leader_election.cc:302] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:34.088799 7470 raft_consensus.cc:2781] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:34.088972 5771 leader_election.cc:302] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:34.089725 7558 raft_consensus.cc:2781] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:34.090337 7558 raft_consensus.cc:686] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:34.090962 5772 leader_election.cc:302] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:34.091585 7560 raft_consensus.cc:2781] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:34.091884 7560 raft_consensus.cc:482] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:34.092247 7560 raft_consensus.cc:3037] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:34.092700 7583 consensus_queue.cc:1035] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:34.093415 7470 raft_consensus.cc:686] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:34.094285 7470 consensus_queue.cc:237] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.096032 7558 consensus_queue.cc:237] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:34.101496 6046 tablet_bootstrap.cc:492] T 8514ca242bb2418d8ddbb88548b21832 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:34.101897 6046 ts_tablet_manager.cc:1397] T 8514ca242bb2418d8ddbb88548b21832 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.183s user 0.010s sys 0.024s
I20260808 13:35:34.106093 7560 raft_consensus.cc:504] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.108332 7560 leader_election.cc:288] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:34.113600 6026 tablet_bootstrap.cc:492] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:34.114100 6026 ts_tablet_manager.cc:1397] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.325s user 0.015s sys 0.002s
W20260808 13:35:34.114286 5771 leader_election.cc:341] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.115361 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1847d0fecb843cdbc833ad674a503a4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:34.116780 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9423b380a62c467895e7fdff14756586" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:34.117237 5383 raft_consensus.cc:3037] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:34.114709 5771 leader_election.cc:302] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:34.124049 5551 leader_election.cc:341] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.125442 5383 raft_consensus.cc:2454] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:34.126752 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 70ec24d1bd3a42b0af377a6461f9264d (DEFAULT_TABLE table=default.Table_7e9aac444b5a43f29dc7608e6ef379f3 [id=6a2ea5033865443d85ac6e6429054471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.127777 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ec24d1bd3a42b0af377a6461f9264d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.138510 7199 ts_tablet_manager.cc:1918] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:34.143435 7199 log.cc:1196] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/a8e18e51c2e24d3caa3d29e6645e46ae
I20260808 13:35:34.144320 7199 ts_tablet_manager.cc:1939] T a8e18e51c2e24d3caa3d29e6645e46ae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:34.104070 6046 raft_consensus.cc:348] T 8514ca242bb2418d8ddbb88548b21832 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.154588 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_dae35fadbf624671ad716102b5182446" } new_table_name: "default.Table_c2023a99a0654190adb8ada69a32f740" modify_external_catalogs: true
I20260808 13:35:34.149863 6046 raft_consensus.cc:374] T 8514ca242bb2418d8ddbb88548b21832 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:34.155237 6046 raft_consensus.cc:729] T 8514ca242bb2418d8ddbb88548b21832 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:34.155859 6046 consensus_queue.cc:260] T 8514ca242bb2418d8ddbb88548b21832 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:34.156931 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0b316390787f4170956f1c47a525d430 (table default.Table_f3144416bf1140aaaa78d9ec41319c6b [id=abda1a6df047407ca80dadf6d4ccb804], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0b316390787f4170956f1c47a525d430 consensus state has no leader
I20260808 13:35:34.165663 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_8e189703ccf14f70903828148ce964df" } modify_external_catalogs: true
I20260808 13:35:34.175873 5338 tablet_service.cc:1467] Processing CreateTablet for tablet e5fb1d373c4d486ca3815023897721a2 (DEFAULT_TABLE table=default.Table_2a111b21bed74a8bb3c9822dff0c2cc6 [id=6a8b8e54b4b24c0c84ac855dea6afd49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.177767 5338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fb1d373c4d486ca3815023897721a2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:34.179571 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 65327us
I20260808 13:35:34.179634 7583 raft_consensus.cc:2726] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:34.180202 5771 leader_election.cc:341] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.180585 5771 leader_election.cc:302] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:34.181490 7241 raft_consensus.cc:2726] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:34.182272 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 83889.3us
I20260808 13:35:34.184347 6033 ts_tablet_manager.cc:1428] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.175s user 0.006s sys 0.001s
I20260808 13:35:34.185439 6033 tablet_bootstrap.cc:492] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:34.085860 7434 leader_election.cc:288] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:34.197389 6026 raft_consensus.cc:348] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.198159 6026 raft_consensus.cc:374] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:34.198484 6026 raft_consensus.cc:729] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:34.203651 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 272) took 5782 ms. Trace:
I20260808 13:35:34.203904 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_8e189703ccf14f70903828148ce964df" } modify_external_catalogs: true
I20260808 13:35:34.203998 7199 rpcz_store.cc:276] 0808 13:35:28.421639 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.421776 (+ 137us) service_pool.cc:227] Handling call
0808 13:35:33.845693 (+5423917us) ts_tablet_manager.cc:1145] Deleting tablet a8e18e51c2e24d3caa3d29e6645e46ae
0808 13:35:33.845741 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.986519 (+140778us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.138260 (+151741us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.203581 (+ 65321us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":41073,"tablet-delete.queue_time_us":5423376}
W20260808 13:35:34.193156 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 30691adffb1f40bdaeaa24f978a38bd7 has no consensus state
I20260808 13:35:34.205418 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 273) took 5290 ms. Trace:
I20260808 13:35:34.205760 7199 rpcz_store.cc:276] 0808 13:35:28.915136 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:28.915319 (+ 183us) service_pool.cc:227] Handling call
0808 13:35:34.205008 (+5289689us) ts_tablet_manager.cc:1145] Deleting tablet 9f53e5b5c4bd4a9385ad198b283c6e79
0808 13:35:34.205064 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.205365 (+ 301us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5289066}
I20260808 13:35:34.207540 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet a8e18e51c2e24d3caa3d29e6645e46ae (table default.Table_321b4eb3dd024925af1cb354d3fe1567 [id=3c108c9e89424d9a9e11554601cf427c]) successfully deleted
W20260808 13:35:34.208259 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 9f53e5b5c4bd4a9385ad198b283c6e79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9f53e5b5c4bd4a9385ad198b283c6e79 already in progress: creating tablet
I20260808 13:35:34.208524 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_ff9135dbb8a84d158dc9f2f209d8d741" } modify_external_catalogs: true
I20260808 13:35:34.218065 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_ff9135dbb8a84d158dc9f2f209d8d741" } modify_external_catalogs: true
W20260808 13:35:34.224066 5771 consensus_peers.cc:487] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:34.221318 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 66160.3us
I20260808 13:35:34.231577 7241 consensus_queue.cc:1035] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:34.239210 7199 tablet_replica.cc:330] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:34.240245 7199 raft_consensus.cc:2229] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:34.187964 7547 raft_consensus.cc:482] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 98dd96f425fc4ab3a6a54f2dfaaf9fab)
I20260808 13:35:34.247083 7547 raft_consensus.cc:504] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:34.251562 7199 raft_consensus.cc:2258] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:34.256116 7591 log.cc:924] Time spent T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: Append to log took a long time: real 0.128s user 0.001s sys 0.000s
W20260808 13:35:34.258838 5551 leader_election.cc:341] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.259245 5551 leader_election.cc:302] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:34.259361 7562 raft_consensus.cc:504] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.260049 7434 raft_consensus.cc:2726] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:34.261780 5551 leader_election.cc:341] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.266033 7547 leader_election.cc:288] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:34.273154 7199 ts_tablet_manager.cc:1905] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:34.199115 6026 consensus_queue.cc:260] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.283174 6026 ts_tablet_manager.cc:1428] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.169s user 0.006s sys 0.000s
I20260808 13:35:34.293370 7562 leader_election.cc:288] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:34.297111 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1847d0fecb843cdbc833ad674a503a4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:34.299697 5385 raft_consensus.cc:3037] T 8514ca242bb2418d8ddbb88548b21832 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:34.193011 7013 ts_tablet_manager.cc:1918] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:34.193873 6046 ts_tablet_manager.cc:1428] T 8514ca242bb2418d8ddbb88548b21832 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.092s user 0.005s sys 0.001s
I20260808 13:35:34.313488 6026 tablet_bootstrap.cc:492] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:34.313763 7013 log.cc:1196] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/ddfd9e3f002445738df24ebb0a333440
I20260808 13:35:34.314445 6046 tablet_bootstrap.cc:492] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:34.195358 5285 consensus_peers.cc:487] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:34.274734 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1847d0fecb843cdbc833ad674a503a4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:34.314824 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 119571us
I20260808 13:35:34.318924 7013 ts_tablet_manager.cc:1939] T ddfd9e3f002445738df24ebb0a333440 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:34.220762 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba098c946794607b2a15bc64098d918" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:34.222102 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ba43d1479a4f9e9d08bdfa29e7157b" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:34.319804 5875 raft_consensus.cc:3037] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:34.320116 5958 tablet.cc:2369] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:35:34.222579 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba098c946794607b2a15bc64098d918" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:34.322849 5384 raft_consensus.cc:3037] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:34.331673 5285 leader_election.cc:341] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: e992a0294c404cfb9541b57782a05fc9
I20260808 13:35:34.331290 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba098c946794607b2a15bc64098d918" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:34.333158 5640 raft_consensus.cc:3037] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:34.319844 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 279) took 5148 ms. Trace:
I20260808 13:35:34.343078 7013 rpcz_store.cc:276] 0808 13:35:29.171336 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:29.228164 (+ 56828us) service_pool.cc:227] Handling call
0808 13:35:33.732746 (+4504582us) ts_tablet_manager.cc:1145] Deleting tablet ddfd9e3f002445738df24ebb0a333440
0808 13:35:33.732800 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.102744 (+369944us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.192752 (+ 90008us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.319795 (+127043us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4483470}
I20260808 13:35:34.345419 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 285) took 4619 ms. Trace:
I20260808 13:35:34.283113 7014 ts_tablet_manager.cc:1918] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:34.350919 7014 log.cc:1196] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/31fc9fdb5ef543da833f9af6ca5807a9
I20260808 13:35:34.351883 7014 ts_tablet_manager.cc:1939] T 31fc9fdb5ef543da833f9af6ca5807a9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:34.354259 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 275) took 4527 ms. Trace:
I20260808 13:35:34.357378 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]) successfully deleted
W20260808 13:35:34.358946 5771 leader_election.cc:341] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.360023 5771 leader_election.cc:302] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:34.370095 5771 leader_election.cc:341] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.387545 7558 raft_consensus.cc:482] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:34.389434 7558 raft_consensus.cc:504] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.346082 7013 rpcz_store.cc:276] 0808 13:35:29.725653 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:29.725770 (+ 117us) service_pool.cc:227] Handling call
0808 13:35:34.344499 (+4618729us) ts_tablet_manager.cc:1145] Deleting tablet 0756b5f24ed94a9fbe2a43efc7b8d40d
0808 13:35:34.344748 (+ 249us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.345300 (+ 552us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4618180}
W20260808 13:35:34.392717 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0756b5f24ed94a9fbe2a43efc7b8d40d already in progress: creating tablet
W20260808 13:35:34.360741 5551 leader_election.cc:341] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.373104 6026 tablet_bootstrap.cc:654] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:34.401645 7014 rpcz_store.cc:276] 0808 13:35:29.826786 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:29.829918 (+ 3132us) service_pool.cc:227] Handling call
0808 13:35:33.791790 (+3961872us) ts_tablet_manager.cc:1145] Deleting tablet 31fc9fdb5ef543da833f9af6ca5807a9
0808 13:35:33.904893 (+113103us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b5800001898. stack: 00007f5a54110033 00007f5a5410ff30 00007f5a53c91534 00000000004db2b4 00000000004db249 00007f5a53f86f11 00007f5a5410a851 00007f5a5410ae82 00007f5a5cfca8b9 00007f5a5bffe5d8 00007f5a5c0c2389 00007f5a5c0c2983 00007f5a5c0f2e10 00007f5a5c0d9e0d 00007f5a5c0d9db9 00007f5a5c0d9d49
0808 13:35:33.904950 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:33.948656 (+ 43706us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.282842 (+334186us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.354193 (+ 71351us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9655680,"tablet-delete.queue_time_us":3961193}
I20260808 13:35:34.403165 7014 tablet_replica.cc:330] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:34.404682 7014 raft_consensus.cc:2229] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:34.405174 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 31fc9fdb5ef543da833f9af6ca5807a9 (table default.Table_faaabef2a0ab42648427c33cda679fb7 [id=7afc114b9ab641df8c540f834036b70c]) successfully deleted
I20260808 13:35:34.394356 7558 leader_election.cc:288] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:34.411332 7013 tablet_replica.cc:330] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:34.427641 7013 raft_consensus.cc:2229] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:34.428745 6033 tablet_bootstrap.cc:654] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:34.428880 7013 raft_consensus.cc:2258] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:34.432446 7013 ts_tablet_manager.cc:1905] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:34.441385 7014 raft_consensus.cc:2258] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:34.442039 7561 raft_consensus.cc:2726] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:34.454097 6026 tablet_bootstrap.cc:492] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:34.454677 6026 ts_tablet_manager.cc:1397] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.141s user 0.010s sys 0.014s
I20260808 13:35:34.464972 7470 consensus_queue.cc:1035] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:34.437786 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7284afe0294644a368cf51451c7ab6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:34.474944 6026 raft_consensus.cc:348] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.476064 6026 raft_consensus.cc:374] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:34.476368 6026 raft_consensus.cc:729] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:34.477124 6026 consensus_queue.cc:260] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.464388 5640 raft_consensus.cc:2454] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:34.444754 7014 ts_tablet_manager.cc:1905] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:34.488039 6026 ts_tablet_manager.cc:1428] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.033s user 0.005s sys 0.003s
W20260808 13:35:34.493731 5283 leader_election.cc:341] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: e992a0294c404cfb9541b57782a05fc9
I20260808 13:35:34.494212 5283 leader_election.cc:302] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:34.487962 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7284afe0294644a368cf51451c7ab6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:34.497838 6026 tablet_bootstrap.cc:492] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:34.515425 7568 raft_consensus.cc:2726] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:34.523118 6026 tablet_bootstrap.cc:654] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:34.529596 6046 tablet_bootstrap.cc:654] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:34.530869 5875 raft_consensus.cc:2454] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:34.533653 5550 leader_election.cc:302] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:34.535285 7470 raft_consensus.cc:2781] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:34.536305 7470 raft_consensus.cc:686] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:34.537714 7470 consensus_queue.cc:237] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:34.567916 7619 log.cc:924] Time spent T 8514ca242bb2418d8ddbb88548b21832 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.071s user 0.002s sys 0.000s
I20260808 13:35:34.537112 5772 leader_election.cc:302] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:34.588109 6026 tablet_bootstrap.cc:492] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:34.588572 6026 ts_tablet_manager.cc:1397] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.091s user 0.012s sys 0.006s
I20260808 13:35:34.593247 6026 raft_consensus.cc:348] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.594282 6026 raft_consensus.cc:374] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260808 13:35:34.600347 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 63313.9us
W20260808 13:35:34.601785 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 185778us
I20260808 13:35:34.594563 6026 raft_consensus.cc:729] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:34.602825 6026 consensus_queue.cc:260] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.613240 7583 raft_consensus.cc:2781] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:34.614199 7583 raft_consensus.cc:686] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:34.624328 6026 ts_tablet_manager.cc:1428] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.035s user 0.008s sys 0.003s
I20260808 13:35:34.627149 6026 tablet_bootstrap.cc:492] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:34.601418 5772 leader_election.cc:341] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:34.634642 5731 tablet.cc:2369] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:35:34.635550 5771 leader_election.cc:341] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.615223 7583 consensus_queue.cc:237] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:34.652740 7633 log.cc:924] Time spent T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.079s user 0.002s sys 0.000s
I20260808 13:35:34.657171 5771 leader_election.cc:302] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:34.665879 6046 tablet_bootstrap.cc:492] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:34.666368 6046 ts_tablet_manager.cc:1397] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.352s user 0.011s sys 0.018s
I20260808 13:35:34.682236 7624 raft_consensus.cc:2726] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:34.688933 5600 tablet_service.cc:1467] Processing CreateTablet for tablet c8cc2a1b87b14d29838bbbe3de44d2a1 (DEFAULT_TABLE table=default.Table_fb8b49faec0a429d95f595b94f65c8e1 [id=530d1358ac09461085dc32e7e952a79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.690286 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8cc2a1b87b14d29838bbbe3de44d2a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.695461 5826 tablet_service.cc:1467] Processing CreateTablet for tablet c8cc2a1b87b14d29838bbbe3de44d2a1 (DEFAULT_TABLE table=default.Table_fb8b49faec0a429d95f595b94f65c8e1 [id=530d1358ac09461085dc32e7e952a79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.696898 5826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8cc2a1b87b14d29838bbbe3de44d2a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.709677 4837 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:34.712370 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 658368f1cd8d470c83c613f9697204ee (DEFAULT_TABLE table=default.Table_5f0b33d3ad8942c0adb8ded9e15220c3 [id=f432910006cd459b9c1c110e9a4a3bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.713543 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 658368f1cd8d470c83c613f9697204ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.720777 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 658368f1cd8d470c83c613f9697204ee (DEFAULT_TABLE table=default.Table_5f0b33d3ad8942c0adb8ded9e15220c3 [id=f432910006cd459b9c1c110e9a4a3bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.721809 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 658368f1cd8d470c83c613f9697204ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.726529 7199 ts_tablet_manager.cc:1918] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:35:34.726303 7651 tablet.cc:1722] T b8ea98afef6841fead641b314ee53a6e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:34.731272 5592 tablet_service.cc:1467] Processing CreateTablet for tablet 433fcbc6cdca458b8404ae5862f280a3 (DEFAULT_TABLE table=default.Table_e4b2b77a6c9343b39de382dcd433c9e7 [id=6fdd797fbbf94dafa4f0048d8f4fb099]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.732371 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 433fcbc6cdca458b8404ae5862f280a3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:34.737411 7644 log.cc:924] Time spent T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.051s user 0.001s sys 0.000s
I20260808 13:35:34.694720 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 658368f1cd8d470c83c613f9697204ee (DEFAULT_TABLE table=default.Table_5f0b33d3ad8942c0adb8ded9e15220c3 [id=f432910006cd459b9c1c110e9a4a3bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.695883 5337 tablet_service.cc:1467] Processing CreateTablet for tablet c8cc2a1b87b14d29838bbbe3de44d2a1 (DEFAULT_TABLE table=default.Table_fb8b49faec0a429d95f595b94f65c8e1 [id=530d1358ac09461085dc32e7e952a79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.721488 5334 tablet_service.cc:1467] Processing CreateTablet for tablet 433fcbc6cdca458b8404ae5862f280a3 (DEFAULT_TABLE table=default.Table_e4b2b77a6c9343b39de382dcd433c9e7 [id=6fdd797fbbf94dafa4f0048d8f4fb099]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.739332 7649 tablet.cc:1722] T b8ea98afef6841fead641b314ee53a6e P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:34.740556 6046 raft_consensus.cc:348] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.741324 6046 raft_consensus.cc:374] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:34.741575 6046 raft_consensus.cc:729] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:34.758111 5827 tablet_service.cc:1467] Processing CreateTablet for tablet 433fcbc6cdca458b8404ae5862f280a3 (DEFAULT_TABLE table=default.Table_e4b2b77a6c9343b39de382dcd433c9e7 [id=6fdd797fbbf94dafa4f0048d8f4fb099]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.759491 5827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 433fcbc6cdca458b8404ae5862f280a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.759941 4837 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:34.760993 5338 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec2c581cf62481789171b7a15512fa4 (DEFAULT_TABLE table=default.Table_43f5211ad48d49e982e3ac2625080359 [id=0111d9bdd1f045b197b24461bc37070f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.766335 7014 ts_tablet_manager.cc:1918] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:34.766724 7014 log.cc:1196] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/2cd6945ecccc4684a3fa5b2a1f07057f
I20260808 13:35:34.767566 7014 ts_tablet_manager.cc:1939] T 2cd6945ecccc4684a3fa5b2a1f07057f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:34.770654 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 658368f1cd8d470c83c613f9697204ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.772471 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 433fcbc6cdca458b8404ae5862f280a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.772652 5598 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec2c581cf62481789171b7a15512fa4 (DEFAULT_TABLE table=default.Table_43f5211ad48d49e982e3ac2625080359 [id=0111d9bdd1f045b197b24461bc37070f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.773883 5598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec2c581cf62481789171b7a15512fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.774674 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8cc2a1b87b14d29838bbbe3de44d2a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.777048 7013 ts_tablet_manager.cc:1918] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:34.777520 7013 log.cc:1196] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/31fc9fdb5ef543da833f9af6ca5807a9
I20260808 13:35:34.778563 7013 ts_tablet_manager.cc:1939] T 31fc9fdb5ef543da833f9af6ca5807a9 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:34.782007 7562 raft_consensus.cc:482] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:34.782724 7562 raft_consensus.cc:504] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.784034 7470 raft_consensus.cc:482] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:34.784682 7470 raft_consensus.cc:504] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.788025 7562 leader_election.cc:288] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:34.788935 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7284afe0294644a368cf51451c7ab6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:34.789875 5875 raft_consensus.cc:2454] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:34.793656 5829 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec2c581cf62481789171b7a15512fa4 (DEFAULT_TABLE table=default.Table_43f5211ad48d49e982e3ac2625080359 [id=0111d9bdd1f045b197b24461bc37070f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.795459 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec2c581cf62481789171b7a15512fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.791940 5550 leader_election.cc:302] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:34.822214 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 1f0a6d9bd2c74d0790badd7977f027e3
I20260808 13:35:34.726909 7199 log.cc:1196] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/ddfd9e3f002445738df24ebb0a333440
I20260808 13:35:34.832013 7199 ts_tablet_manager.cc:1939] T ddfd9e3f002445738df24ebb0a333440 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:34.839035 7434 raft_consensus.cc:2781] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:34.839721 7434 raft_consensus.cc:482] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:34.840327 7434 raft_consensus.cc:3037] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:34.843051 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 6e70a5cf88d94b36913a6d1ccfcb979a
I20260808 13:35:34.845362 5610 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f0a6d9bd2c74d0790badd7977f027e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:34 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:34.853809 5610 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e70a5cf88d94b36913a6d1ccfcb979a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:34 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:34.856104 7470 leader_election.cc:288] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:34.794821 6046 consensus_queue.cc:260] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.800118 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7284afe0294644a368cf51451c7ab6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:34.800040 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 2d4a2d7ddb45419bbbe5df01a5e1d40f (DEFAULT_TABLE table=default.Table_fbaf65bcf1fd48249596df8d79af088b [id=b529cc209a934207ab4940d51ee4876b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.865321 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 287) took 5040 ms. Trace:
I20260808 13:35:34.865674 7013 rpcz_store.cc:276] 0808 13:35:29.824802 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:29.824983 (+ 181us) service_pool.cc:227] Handling call
0808 13:35:34.391606 (+4566623us) ts_tablet_manager.cc:1145] Deleting tablet 31fc9fdb5ef543da833f9af6ca5807a9
0808 13:35:34.411168 (+ 19562us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.591295 (+180127us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.776825 (+185530us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.865245 (+ 88420us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4552885}
I20260808 13:35:34.868528 6046 ts_tablet_manager.cc:1428] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.130s user 0.003s sys 0.005s
I20260808 13:35:34.825177 5325 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:46324 (request call id 33) took 1077 ms. Trace:
I20260808 13:35:34.870873 5325 rpcz_store.cc:276] 0808 13:35:33.746557 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.755929 (+ 9372us) service_pool.cc:227] Handling call
0808 13:35:34.290702 (+534773us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b5400111554. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c27a25268 00007f5c28a116cd 00007f5c272b9e04 00007f5c272b9d92 00007f5c272b9cc1 00007f5c272b9c3c 00007f5c272b8ef2 00007f5c257da94c 00007f5c257d9d86 00007f5c257d972f
0808 13:35:34.551036 (+260334us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54000c2954. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c27a25268 00007f5c28a116cd 00007f5c272b9e04 00007f5c272b9d92 00007f5c272b9cc1 00007f5c272b9c3c 00007f5c272b8ef2 00007f5c257da94c 00007f5c257d9d86 00007f5c257d972f
0808 13:35:34.824253 (+273217us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":709827584}
I20260808 13:35:34.830822 5338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec2c581cf62481789171b7a15512fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.839649 5335 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f0a6d9bd2c74d0790badd7977f027e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:34 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:34.853663 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e70a5cf88d94b36913a6d1ccfcb979a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:34 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:34.869923 5551 leader_election.cc:341] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.879004 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d4a2d7ddb45419bbbe5df01a5e1d40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.879699 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 276) took 4652 ms. Trace:
I20260808 13:35:34.881868 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 31fc9fdb5ef543da833f9af6ca5807a9 (table default.Table_faaabef2a0ab42648427c33cda679fb7 [id=7afc114b9ab641df8c540f834036b70c]) successfully deleted
I20260808 13:35:34.883242 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 288) took 4638 ms. Trace:
I20260808 13:35:34.880043 7014 rpcz_store.cc:276] 0808 13:35:30.226998 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.227144 (+ 146us) service_pool.cc:227] Handling call
0808 13:35:34.403020 (+4175876us) ts_tablet_manager.cc:1145] Deleting tablet 2cd6945ecccc4684a3fa5b2a1f07057f
0808 13:35:34.403061 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.597557 (+194496us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.766076 (+168519us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.879632 (+113556us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4175257}
I20260808 13:35:34.862077 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66b38e83dab4bf1a3d2d1e088a96988" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:34.889294 5551 leader_election.cc:341] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.890374 7013 rpcz_store.cc:276] 0808 13:35:30.246387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.246650 (+ 263us) service_pool.cc:227] Handling call
0808 13:35:34.882703 (+4636053us) ts_tablet_manager.cc:1145] Deleting tablet 973a7afdcbf944aab8f11b3fd3995865
0808 13:35:34.882826 (+ 123us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.883180 (+ 354us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4635316}
I20260808 13:35:34.891875 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 277) took 4540 ms. Trace:
W20260808 13:35:34.893761 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 973a7afdcbf944aab8f11b3fd3995865 because the tablet was not found. No further retry: Not found: Tablet not found: 973a7afdcbf944aab8f11b3fd3995865
I20260808 13:35:34.896931 5831 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e70a5cf88d94b36913a6d1ccfcb979a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:34 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:34.898180 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 2cd6945ecccc4684a3fa5b2a1f07057f (table default.Table_b75082dac94348fab668160895573637 [id=56f4695998f34301b8528a79b16e2dd2]) successfully deleted
I20260808 13:35:34.898794 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f0a6d9bd2c74d0790badd7977f027e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:34 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:34.900199 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66b38e83dab4bf1a3d2d1e088a96988" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:34.910844 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 289) took 4662 ms. Trace:
W20260808 13:35:34.902138 5550 leader_election.cc:341] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:34.902323 7434 raft_consensus.cc:504] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:34.914971 5550 leader_election.cc:302] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:34.916359 7014 rpcz_store.cc:276] 0808 13:35:30.351051 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.351205 (+ 154us) service_pool.cc:227] Handling call
0808 13:35:34.891507 (+4540302us) ts_tablet_manager.cc:1145] Deleting tablet ba39706cff65404485959865fff1772a
0808 13:35:34.891555 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.891814 (+ 259us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4539667}
W20260808 13:35:34.924075 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet ba39706cff65404485959865fff1772a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba39706cff65404485959865fff1772a already in progress: creating tablet
I20260808 13:35:34.924213 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:34.924880 7562 raft_consensus.cc:2726] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:34.926234 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:34.929158 7434 leader_election.cc:288] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:34.933135 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7284afe0294644a368cf51451c7ab6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:34.933820 5882 raft_consensus.cc:3037] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:34.935405 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 274) took 5766 ms. Trace:
I20260808 13:35:34.935819 7199 rpcz_store.cc:276] 0808 13:35:29.168844 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:29.168988 (+ 144us) service_pool.cc:227] Handling call
0808 13:35:34.206537 (+5037549us) ts_tablet_manager.cc:1145] Deleting tablet ddfd9e3f002445738df24ebb0a333440
0808 13:35:34.238974 (+ 32437us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.502493 (+263519us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.726255 (+223762us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:34.935341 (+209086us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4890395}
I20260808 13:35:34.940883 7583 consensus_queue.cc:1035] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:34.944464 7013 rpcz_store.cc:276] 0808 13:35:30.247990 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.248431 (+ 441us) service_pool.cc:227] Handling call
0808 13:35:34.910330 (+4661899us) ts_tablet_manager.cc:1145] Deleting tablet 81fb3e099f7f4544b3c100070f0d14ef
0808 13:35:34.910415 (+ 85us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.910774 (+ 359us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4661210}
I20260808 13:35:34.919078 6046 tablet_bootstrap.cc:492] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:34.937229 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7284afe0294644a368cf51451c7ab6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:34.937572 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet ddfd9e3f002445738df24ebb0a333440 (table default.Table_40a75c335146479fb7dc85888769c778 [id=39564f6d22d347c7ac176853de55b1ce]) successfully deleted
W20260808 13:35:34.951354 5551 leader_election.cc:341] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:34.946214 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 81fb3e099f7f4544b3c100070f0d14ef because the tablet was not found. No further retry: Not found: Tablet not found: 81fb3e099f7f4544b3c100070f0d14ef
I20260808 13:35:34.951637 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:34.952885 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:34.953168 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 278) took 4439 ms. Trace:
I20260808 13:35:34.954068 4838 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_9d2867cdebf84440b7aa21a71faba9c4 [id=0ddf25cf69f449cf8c7516cb84736b3d] alter complete (version 2)
I20260808 13:35:34.953843 7014 rpcz_store.cc:276] 0808 13:35:30.514207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.514354 (+ 147us) service_pool.cc:227] Handling call
0808 13:35:34.952772 (+4438418us) ts_tablet_manager.cc:1145] Deleting tablet 9ea8f5322a404837b91e1d96749926b7
0808 13:35:34.952828 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.953101 (+ 273us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4437693}
I20260808 13:35:34.960542 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 2d4a2d7ddb45419bbbe5df01a5e1d40f (DEFAULT_TABLE table=default.Table_fbaf65bcf1fd48249596df8d79af088b [id=b529cc209a934207ab4940d51ee4876b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.962850 4840 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_e101c061f2bf4821b6fe2c6108432735 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7] alter complete (version 3)
I20260808 13:35:34.968611 5831 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e18e51c2e24d3caa3d29e6645e46ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:34.969384 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:25 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:34.971557 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9bc338093f403881d329a7e913c9f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:20 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:34.971570 5831 tablet_service.cc:1514] Processing DeleteTablet for tablet 127e8d0323954ef788018bcb91582951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:34.972150 4840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 35) took 3784 ms (client timeout 1000 ms). Trace:
I20260808 13:35:34.972703 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet 9423b380a62c467895e7fdff14756586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:34.974344 6033 tablet_bootstrap.cc:492] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:34.974889 6033 ts_tablet_manager.cc:1397] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.790s user 0.022s sys 0.012s
I20260808 13:35:34.904335 6026 tablet_bootstrap.cc:654] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:34.979318 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 9be987112b0c4fa889d5458c65089098 (DEFAULT_TABLE table=default.Table_0ab0a38241284050a12358d0f8d76032 [id=b713080d065a4fa49a583244a722909c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.980358 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be987112b0c4fa889d5458c65089098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.984915 5335 tablet_service.cc:1467] Processing CreateTablet for tablet 205532710f0148f38d12a485bc0107ec (DEFAULT_TABLE table=default.Table_ec97d7d5adc245a6b9d4224e8ab75cd5 [id=fd9c3138231f42b791685b0df92ac968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.985281 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 9be987112b0c4fa889d5458c65089098 (DEFAULT_TABLE table=default.Table_0ab0a38241284050a12358d0f8d76032 [id=b713080d065a4fa49a583244a722909c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.902925 7470 consensus_queue.cc:1035] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:34.987049 5335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 205532710f0148f38d12a485bc0107ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:34.990094 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 9be987112b0c4fa889d5458c65089098 (DEFAULT_TABLE table=default.Table_0ab0a38241284050a12358d0f8d76032 [id=b713080d065a4fa49a583244a722909c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:34.991410 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be987112b0c4fa889d5458c65089098. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:34.996008 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 9ea8f5322a404837b91e1d96749926b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ea8f5322a404837b91e1d96749926b7 already in progress: creating tablet
I20260808 13:35:34.997916 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 284) took 3617 ms. Trace:
I20260808 13:35:34.998313 7014 rpcz_store.cc:276] 0808 13:35:31.380517 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:31.386708 (+ 6191us) service_pool.cc:227] Handling call
0808 13:35:34.997476 (+3610768us) ts_tablet_manager.cc:1145] Deleting tablet 3272708a835c432c878cc330e44490c5
0808 13:35:34.997540 (+ 64us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.997839 (+ 299us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3610186}
W20260808 13:35:34.999796 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 3272708a835c432c878cc330e44490c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3272708a835c432c878cc330e44490c5 already in progress: creating tablet
I20260808 13:35:34.988445 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 290) took 4738 ms. Trace:
I20260808 13:35:35.008343 7013 rpcz_store.cc:276] 0808 13:35:30.250353 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.250459 (+ 106us) service_pool.cc:227] Handling call
0808 13:35:34.987927 (+4737468us) ts_tablet_manager.cc:1145] Deleting tablet 45a26598a69146e1950e9227523fff13
0808 13:35:34.987983 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:34.988389 (+ 406us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4736883}
I20260808 13:35:34.978967 7434 raft_consensus.cc:482] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:35.009459 7434 raft_consensus.cc:504] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.011722 5882 raft_consensus.cc:2454] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:35.011824 7434 leader_election.cc:288] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:35.013015 5600 tablet_service.cc:1467] Processing CreateTablet for tablet 2d4a2d7ddb45419bbbe5df01a5e1d40f (DEFAULT_TABLE table=default.Table_fbaf65bcf1fd48249596df8d79af088b [id=b529cc209a934207ab4940d51ee4876b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.014318 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d4a2d7ddb45419bbbe5df01a5e1d40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.016296 5550 leader_election.cc:302] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:35.017066 7664 tablet.cc:1722] T b8ea98afef6841fead641b314ee53a6e P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:35.017808 7562 raft_consensus.cc:2781] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:35.018245 7562 raft_consensus.cc:686] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:35.018999 7562 consensus_queue.cc:237] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.024009 5606 tablet_service.cc:1467] Processing CreateTablet for tablet 205532710f0148f38d12a485bc0107ec (DEFAULT_TABLE table=default.Table_ec97d7d5adc245a6b9d4224e8ab75cd5 [id=fd9c3138231f42b791685b0df92ac968]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:35.028708 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 45a26598a69146e1950e9227523fff13 because the tablet was not found. No further retry: Not found: Tablet not found: 45a26598a69146e1950e9227523fff13
I20260808 13:35:35.029085 6046 tablet_bootstrap.cc:654] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:35.024479 7583 raft_consensus.cc:482] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:35.029970 7583 raft_consensus.cc:504] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.031942 7583 leader_election.cc:288] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:35.031699 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264ee45f8dfa4a05b0e5e812ce1b622c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:35.034991 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0cc688386f3413ea939dda3a75d6541" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:34.972596 4840 rpcz_store.cc:269] 0808 13:35:31.187460 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:31.187626 (+ 166us) service_pool.cc:227] Handling call
0808 13:35:34.972035 (+3784409us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:31.840928 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:31.841283 (+ 355us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:31.841307 (+ 24us) write_op.cc:435] Acquired schema lock
0808 13:35:31.841320 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:31.841629 (+ 309us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:31.841663 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:31.841684 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:31.841970 (+ 286us) tablet.cc:699] Row locks acquired
0808 13:35:31.841985 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:31.842089 (+ 104us) write_op.cc:270] Start()
0808 13:35:31.842167 (+ 78us) write_op.cc:276] Timestamp: P: 1786196131840733 usec, L: 0
0808 13:35:31.842182 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:31.842764 (+ 582us) log.cc:842] Serialized 901 byte log entry
0808 13:35:32.013443 (+170679us) op_driver.cc:444] REPLICATION: finished
0808 13:35:32.014866 (+ 1423us) write_op.cc:301] APPLY: starting
0808 13:35:32.014981 (+ 115us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:32.015212 (+ 231us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:32.015690 (+ 478us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:32.016651 (+ 961us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:32.016939 (+ 288us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:32.017034 (+ 95us) write_op.cc:312] APPLY: finished
0808 13:35:32.017957 (+ 923us) log.cc:842] Serialized 29 byte log entry
0808 13:35:32.018398 (+ 441us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:32.018553 (+ 155us) write_op.cc:454] Released schema lock
0808 13:35:32.018651 (+ 98us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:32.022940 (+ 4289us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:32.023553 (+ 613us) write_op.cc:454] Released schema lock
Related trace 'op':
0808 13:35:32.370244 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:32.370600 (+ 356us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:32.370656 (+ 56us) write_op.cc:435] Acquired schema lock
0808 13:35:32.370667 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:32.370914 (+ 247us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:32.370947 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:32.370969 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:32.371096 (+ 127us) tablet.cc:699] Row locks acquired
0808 13:35:32.371109 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:32.371205 (+ 96us) write_op.cc:270] Start()
0808 13:35:32.371279 (+ 74us) write_op.cc:276] Timestamp: P: 1786196132371167 usec, L: 0
0808 13:35:32.371294 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:32.371713 (+ 419us) log.cc:842] Serialized 356 byte log entry
0808 13:35:32.402982 (+ 31269us) op_driver.cc:444] REPLICATION: finished
0808 13:35:32.406032 (+ 3050us) write_op.cc:301] APPLY: starting
0808 13:35:32.406084 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:32.406173 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:32.406276 (+ 103us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:32.406418 (+ 142us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:32.406537 (+ 119us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:32.406577 (+ 40us) write_op.cc:312] APPLY: finished
0808 13:35:32.409011 (+ 2434us) log.cc:842] Serialized 23 byte log entry
0808 13:35:32.409138 (+ 127us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:32.409205 (+ 67us) write_op.cc:454] Released schema lock
0808 13:35:32.409235 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:32.409394 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:32.409418 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":617,"apply.run_cpu_time_us":3730,"apply.run_wall_time_us":9153,"num_ops":2,"prepare.queue_time_us":12848,"prepare.run_cpu_time_us":1945,"prepare.run_wall_time_us":2022,"replication_time_us":170346}],["op",{"apply.queue_time_us":2932,"apply.run_cpu_time_us":1320,"apply.run_wall_time_us":3526,"num_ops":1,"prepare.queue_time_us":236688,"prepare.run_cpu_time_us":1819,"prepare.run_wall_time_us":41530,"raft.queue_time_us":299,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":233,"replication_time_us":31617}]]}
W20260808 13:35:35.037782 5771 leader_election.cc:341] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:35.007527 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 288) took 3317 ms. Trace:
I20260808 13:35:35.038864 7014 rpcz_store.cc:276] 0808 13:35:31.689718 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:31.689874 (+ 156us) service_pool.cc:227] Handling call
0808 13:35:35.006968 (+3317094us) ts_tablet_manager.cc:1145] Deleting tablet 62d68ab73bae4dd9af46e907b1fd198c
0808 13:35:35.007010 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.007382 (+ 372us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3307825}
I20260808 13:35:35.039878 7014 tablet_replica.cc:330] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
W20260808 13:35:35.040822 5551 leader_election.cc:341] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:35.040817 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0cc688386f3413ea939dda3a75d6541" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:35.041611 5635 raft_consensus.cc:2454] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:35.042508 5772 leader_election.cc:302] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:35.043675 7583 raft_consensus.cc:2781] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:35.043987 7583 raft_consensus.cc:482] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:35.044286 7583 raft_consensus.cc:3037] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:35.046797 7434 consensus_queue.cc:1035] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:35.048714 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 205532710f0148f38d12a485bc0107ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.076107 6033 raft_consensus.cc:348] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.077220 6033 raft_consensus.cc:374] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:35.077564 6033 raft_consensus.cc:729] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
W20260808 13:35:35.083926 5484 tablet.cc:2369] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:35:35.092056 7014 raft_consensus.cc:2229] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:35.092648 7014 raft_consensus.cc:2258] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:35.095299 7014 ts_tablet_manager.cc:1905] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:35.089157 5772 consensus_peers.cc:487] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2cd6945ecccc4684a3fa5b2a1f07057f. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:35.113004 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 280) took 5470 ms. Trace:
I20260808 13:35:35.113399 7199 rpcz_store.cc:276] 0808 13:35:29.642758 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:29.642921 (+ 163us) service_pool.cc:227] Handling call
0808 13:35:34.938207 (+5295286us) ts_tablet_manager.cc:1145] Deleting tablet 0756b5f24ed94a9fbe2a43efc7b8d40d
0808 13:35:35.112388 (+174181us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.112927 (+ 539us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5294730}
I20260808 13:35:35.115073 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d4a2d7ddb45419bbbe5df01a5e1d40f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:35.115144 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 62d68ab73bae4dd9af46e907b1fd198c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62d68ab73bae4dd9af46e907b1fd198c already in progress: creating tablet
I20260808 13:35:35.120445 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be987112b0c4fa889d5458c65089098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.122731 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264ee45f8dfa4a05b0e5e812ce1b622c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:35.039233 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_b8623fa3d1eb479a96e59b16c329e590"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:35.123383 5882 raft_consensus.cc:2454] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
W20260808 13:35:35.124209 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8623fa3d1eb479a96e59b16c329e590 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:35.128221 5550 leader_election.cc:302] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:35.130980 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0756b5f24ed94a9fbe2a43efc7b8d40d already in progress: creating tablet
I20260808 13:35:35.132368 7199 tablet_replica.cc:330] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:35.133826 7199 raft_consensus.cc:2229] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:35.134572 7434 raft_consensus.cc:2781] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:35.135020 7434 raft_consensus.cc:482] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:35.135646 7434 raft_consensus.cc:3037] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:35.135805 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 24096c0c30da4f2ea643fcdd5791a927 (DEFAULT_TABLE table=default.Table_7cfe2eb0d78341a0816f9b5e14316b4f [id=59ed02534c50457ab73d5c2757bda959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.137699 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24096c0c30da4f2ea643fcdd5791a927. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.149994 5592 tablet_service.cc:1467] Processing CreateTablet for tablet f935e5a12934443aafee878e0af56b9e (DEFAULT_TABLE table=default.Table_bc6082a17c754033966ecce7c4f9bf26 [id=3a6d52daeb034a88bdcf06112086dff8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.151767 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f935e5a12934443aafee878e0af56b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.153942 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 30691adffb1f40bdaeaa24f978a38bd7 (DEFAULT_TABLE table=default.Table_c2023a99a0654190adb8ada69a32f740 [id=054085276211478a8982d48b0599a684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.154985 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30691adffb1f40bdaeaa24f978a38bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.164853 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 2483b3320e2e49db96d39254dd5755aa (DEFAULT_TABLE table=default.Table_291e70645d0a4f24b232391de2e62d9b [id=748566361526422b9dc06be81bcf7286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.166056 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2483b3320e2e49db96d39254dd5755aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.177933 5608 tablet_service.cc:1467] Processing CreateTablet for tablet d0767140736242f18ef4689b112deefb (DEFAULT_TABLE table=default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.179107 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0767140736242f18ef4689b112deefb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.187498 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 24096c0c30da4f2ea643fcdd5791a927 (DEFAULT_TABLE table=default.Table_7cfe2eb0d78341a0816f9b5e14316b4f [id=59ed02534c50457ab73d5c2757bda959]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:35.188854 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0b316390787f4170956f1c47a525d430 (table default.Table_f3144416bf1140aaaa78d9ec41319c6b [id=abda1a6df047407ca80dadf6d4ccb804], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0b316390787f4170956f1c47a525d430 consensus state has no leader
I20260808 13:35:35.147904 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 24096c0c30da4f2ea643fcdd5791a927 (DEFAULT_TABLE table=default.Table_7cfe2eb0d78341a0816f9b5e14316b4f [id=59ed02534c50457ab73d5c2757bda959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.191093 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24096c0c30da4f2ea643fcdd5791a927. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.191428 5340 tablet_service.cc:1467] Processing CreateTablet for tablet f935e5a12934443aafee878e0af56b9e (DEFAULT_TABLE table=default.Table_bc6082a17c754033966ecce7c4f9bf26 [id=3a6d52daeb034a88bdcf06112086dff8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.193370 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 30691adffb1f40bdaeaa24f978a38bd7 (DEFAULT_TABLE table=default.Table_c2023a99a0654190adb8ada69a32f740 [id=054085276211478a8982d48b0599a684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.159873 5831 tablet_service.cc:1467] Processing CreateTablet for tablet f935e5a12934443aafee878e0af56b9e (DEFAULT_TABLE table=default.Table_bc6082a17c754033966ecce7c4f9bf26 [id=3a6d52daeb034a88bdcf06112086dff8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.195919 5831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f935e5a12934443aafee878e0af56b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.197631 6046 tablet_bootstrap.cc:492] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:35.198048 6046 ts_tablet_manager.cc:1397] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.279s user 0.006s sys 0.007s
I20260808 13:35:35.199012 7199 raft_consensus.cc:2258] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:35.200459 6046 raft_consensus.cc:348] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.201028 7199 ts_tablet_manager.cc:1905] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:35.201129 6046 raft_consensus.cc:374] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:35.201362 6046 raft_consensus.cc:729] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:35.172791 5826 tablet_service.cc:1467] Processing CreateTablet for tablet 205532710f0148f38d12a485bc0107ec (DEFAULT_TABLE table=default.Table_ec97d7d5adc245a6b9d4224e8ab75cd5 [id=fd9c3138231f42b791685b0df92ac968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.173599 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 30691adffb1f40bdaeaa24f978a38bd7 (DEFAULT_TABLE table=default.Table_c2023a99a0654190adb8ada69a32f740 [id=054085276211478a8982d48b0599a684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.202474 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30691adffb1f40bdaeaa24f978a38bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.201925 6046 consensus_queue.cc:260] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:35.206804 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 78686.3us
I20260808 13:35:35.210757 6046 ts_tablet_manager.cc:1428] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.012s user 0.007s sys 0.001s
I20260808 13:35:35.213423 5603 tablet_service.cc:1467] Processing CreateTablet for tablet ddd0134819474e7eadd4e3d12d94a7b6 (DEFAULT_TABLE table=default.Table_da0f5516539d4f17b7ad03013cfd9460 [id=b06d9584a664410aad7628dbb2495f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.214542 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddd0134819474e7eadd4e3d12d94a7b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.184572 5832 tablet_service.cc:1467] Processing CreateTablet for tablet d0767140736242f18ef4689b112deefb (DEFAULT_TABLE table=default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.217046 5357 tablet_service.cc:1467] Processing CreateTablet for tablet d0767140736242f18ef4689b112deefb (DEFAULT_TABLE table=default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.218221 5353 tablet_service.cc:1467] Processing CreateTablet for tablet ddd0134819474e7eadd4e3d12d94a7b6 (DEFAULT_TABLE table=default.Table_da0f5516539d4f17b7ad03013cfd9460 [id=b06d9584a664410aad7628dbb2495f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.223209 5337 tablet_service.cc:1467] Processing CreateTablet for tablet 2483b3320e2e49db96d39254dd5755aa (DEFAULT_TABLE table=default.Table_291e70645d0a4f24b232391de2e62d9b [id=748566361526422b9dc06be81bcf7286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.227118 5841 tablet_service.cc:1467] Processing CreateTablet for tablet ddd0134819474e7eadd4e3d12d94a7b6 (DEFAULT_TABLE table=default.Table_da0f5516539d4f17b7ad03013cfd9460 [id=b06d9584a664410aad7628dbb2495f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.227695 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:35.228615 4842 catalog_manager.cc:5495] T 8514ca242bb2418d8ddbb88548b21832 P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:35.221910 7560 consensus_queue.cc:1035] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:35.232102 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 809049a771224082a16c270d01ec4bc7 (DEFAULT_TABLE table=default.Table_dfee9bf8bf9e4bab8982be9ddb880b00 [id=ebdcaa7460264ccf88c51c04111d5c28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.233531 6026 tablet_bootstrap.cc:492] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:35.233933 6026 ts_tablet_manager.cc:1397] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.607s user 0.003s sys 0.016s
I20260808 13:35:35.248479 7013 tablet_replica.cc:330] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:35.249047 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24096c0c30da4f2ea643fcdd5791a927. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.249365 7013 raft_consensus.cc:2229] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:35.249918 7013 raft_consensus.cc:2258] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:35.252728 7013 ts_tablet_manager.cc:1905] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:35.256182 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f935e5a12934443aafee878e0af56b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.257139 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30691adffb1f40bdaeaa24f978a38bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.259441 5598 tablet_service.cc:1467] Processing CreateTablet for tablet 809049a771224082a16c270d01ec4bc7 (DEFAULT_TABLE table=default.Table_dfee9bf8bf9e4bab8982be9ddb880b00 [id=ebdcaa7460264ccf88c51c04111d5c28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.267102 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 2330be58e9d6414da65b4434c2d3b461 (DEFAULT_TABLE table=default.Table_c23b9f32680f412fbc697e136022e69c [id=7b7b344f5dda412389a0223f88351783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.273962 7014 ts_tablet_manager.cc:1918] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:35.274369 7014 log.cc:1196] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/69642fdba1f145438b04eae9f8154101
I20260808 13:35:35.276855 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 52e8243692d847039aafda3e0d0095fb (DEFAULT_TABLE table=default.Table_01763d5cef124bad885eaaba0686787b [id=8269e8c2637f41859e8ead761bc2d7bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.280030 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddd0134819474e7eadd4e3d12d94a7b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.280185 6046 tablet_bootstrap.cc:492] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:35.282919 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0767140736242f18ef4689b112deefb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.283516 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809049a771224082a16c270d01ec4bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.285473 4840 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 392) took 1049 ms. Trace:
I20260808 13:35:35.290283 6026 raft_consensus.cc:348] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.291230 6026 raft_consensus.cc:374] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:35.291540 6026 raft_consensus.cc:729] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:35.230345 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:35.296396 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:35.297423 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:35.300505 5598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809049a771224082a16c270d01ec4bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.301474 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2330be58e9d6414da65b4434c2d3b461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.305560 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2483b3320e2e49db96d39254dd5755aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.309871 5334 tablet_service.cc:1467] Processing CreateTablet for tablet 2330be58e9d6414da65b4434c2d3b461 (DEFAULT_TABLE table=default.Table_c23b9f32680f412fbc697e136022e69c [id=7b7b344f5dda412389a0223f88351783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.312269 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2330be58e9d6414da65b4434c2d3b461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.315148 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 52e8243692d847039aafda3e0d0095fb (DEFAULT_TABLE table=default.Table_01763d5cef124bad885eaaba0686787b [id=8269e8c2637f41859e8ead761bc2d7bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.327131 5385 raft_consensus.cc:3037] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:35.285914 4840 rpcz_store.cc:276] 0808 13:35:34.236145 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.038020 (+801875us) service_pool.cc:227] Handling call
0808 13:35:35.124589 (+ 86569us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:35.142679 (+ 18090us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:35.284333 (+141654us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.284359 (+ 26us) write_op.cc:454] Released schema lock
0808 13:35:35.284564 (+ 205us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:35.285078 (+ 514us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:35.285393 (+ 315us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:35.258189 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:35.258632 (+ 443us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:35.258666 (+ 34us) write_op.cc:435] Acquired schema lock
0808 13:35:35.258686 (+ 20us) tablet.cc:660] Decoding operations
0808 13:35:35.259072 (+ 386us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:35.259124 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:35.259147 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:35.259469 (+ 322us) tablet.cc:699] Row locks acquired
0808 13:35:35.259486 (+ 17us) write_op.cc:260] PREPARE: finished
0808 13:35:35.259625 (+ 139us) write_op.cc:270] Start()
0808 13:35:35.259733 (+ 108us) write_op.cc:276] Timestamp: P: 1786196135259544 usec, L: 0
0808 13:35:35.259752 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:35.260435 (+ 683us) log.cc:842] Serialized 433 byte log entry
0808 13:35:35.272266 (+ 11831us) op_driver.cc:444] REPLICATION: finished
0808 13:35:35.272663 (+ 397us) write_op.cc:301] APPLY: starting
0808 13:35:35.272741 (+ 78us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:35.272845 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:35.272954 (+ 109us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:35.273343 (+ 389us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:35.273478 (+ 135us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:35.273519 (+ 41us) write_op.cc:312] APPLY: finished
0808 13:35:35.273709 (+ 190us) log.cc:842] Serialized 29 byte log entry
0808 13:35:35.273854 (+ 145us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.273925 (+ 71us) write_op.cc:454] Released schema lock
0808 13:35:35.273954 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":231,"apply.run_cpu_time_us":1486,"apply.run_wall_time_us":1485,"num_ops":2,"prepare.queue_time_us":113841,"prepare.run_cpu_time_us":2590,"prepare.run_wall_time_us":2590,"raft.queue_time_us":1907,"raft.run_cpu_time_us":705,"raft.run_wall_time_us":707,"replication_time_us":12423}]]}
I20260808 13:35:35.341533 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e8243692d847039aafda3e0d0095fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.345772 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e8243692d847039aafda3e0d0095fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.295202 6026 consensus_queue.cc:260] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.372572 7014 ts_tablet_manager.cc:1939] T 69642fdba1f145438b04eae9f8154101 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:35.389616 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:35.396450 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:35.397451 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]). Expected version 1 got 0
I20260808 13:35:35.405632 7470 consensus_queue.cc:1035] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:35.361907 7434 raft_consensus.cc:504] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.410267 6046 tablet_bootstrap.cc:654] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:35.103070 6033 consensus_queue.cc:260] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.435150 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 809049a771224082a16c270d01ec4bc7 (DEFAULT_TABLE table=default.Table_dfee9bf8bf9e4bab8982be9ddb880b00 [id=ebdcaa7460264ccf88c51c04111d5c28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.439548 5829 tablet_service.cc:1467] Processing CreateTablet for tablet 2483b3320e2e49db96d39254dd5755aa (DEFAULT_TABLE table=default.Table_291e70645d0a4f24b232391de2e62d9b [id=748566361526422b9dc06be81bcf7286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.094146 7583 raft_consensus.cc:504] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.449152 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:35.450637 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 293) took 3287 ms. Trace:
I20260808 13:35:35.451042 7014 rpcz_store.cc:276] 0808 13:35:32.162963 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.163115 (+ 152us) service_pool.cc:227] Handling call
0808 13:35:35.039709 (+2876594us) ts_tablet_manager.cc:1145] Deleting tablet 69642fdba1f145438b04eae9f8154101
0808 13:35:35.039757 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.104954 (+ 65197us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:35.273724 (+168770us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:35.450543 (+176819us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":51338,"tablet-delete.queue_time_us":2875946}
I20260808 13:35:35.452436 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 69642fdba1f145438b04eae9f8154101 (table default.Table_e0361960d323473f83c14c853ffcb172 [id=e3e121c36ff44ea0a3a98185c8cce381]) successfully deleted
I20260808 13:35:35.464421 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 294) took 3301 ms. Trace:
I20260808 13:35:35.465371 7434 leader_election.cc:288] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:35.467259 6026 ts_tablet_manager.cc:1428] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.233s user 0.004s sys 0.004s
I20260808 13:35:35.468703 6026 tablet_bootstrap.cc:492] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:35.476537 7583 leader_election.cc:288] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:35.476438 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 95979.6us
I20260808 13:35:35.478266 6033 ts_tablet_manager.cc:1428] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.503s user 0.005s sys 0.003s
W20260808 13:35:35.478523 6033 ts_tablet_manager.cc:1460] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Tablet startup took 503ms
I20260808 13:35:35.479449 7568 raft_consensus.cc:482] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:35.478765 6033 ts_tablet_manager.cc:1462] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Trace:
0808 13:35:16.862379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.862506 (+ 127us) service_pool.cc:227] Handling call
0808 13:35:16.995778 (+133272us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:17.028041 (+ 32263us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:17.350189 (+322148us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:17.579653 (+229464us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:17.580148 (+ 495us) inbound_call.cc:173] Queueing success response
0808 13:35:34.185053 (+16604905us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0808 13:35:34.975228 (+790175us) ts_tablet_manager.cc:1429] Starting tablet replica
0808 13:35:34.977632 (+ 2404us) tablet_replica.cc:259] Starting instrumentation
0808 13:35:34.978169 (+ 537us) tablet_replica.cc:278] Starting consensus
Metrics: {"tablet-open.queue_time_us":16605032}
I20260808 13:35:35.479892 7568 raft_consensus.cc:504] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.485667 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba098c946794607b2a15bc64098d918" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:35.486594 5640 raft_consensus.cc:2379] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:35.443550 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 2330be58e9d6414da65b4434c2d3b461 (DEFAULT_TABLE table=default.Table_c23b9f32680f412fbc697e136022e69c [id=7b7b344f5dda412389a0223f88351783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.487685 7568 leader_election.cc:288] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:35.477862 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264ee45f8dfa4a05b0e5e812ce1b622c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:35.464772 7014 rpcz_store.cc:276] 0808 13:35:32.163317 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.164867 (+ 1550us) service_pool.cc:227] Handling call
0808 13:35:35.464069 (+3299202us) ts_tablet_manager.cc:1145] Deleting tablet 8d12106f0d794a79935999a2b8dbe7e9
0808 13:35:35.464117 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.464361 (+ 244us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3298581}
W20260808 13:35:35.491482 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 8d12106f0d794a79935999a2b8dbe7e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d12106f0d794a79935999a2b8dbe7e9 already in progress: creating tablet
I20260808 13:35:35.492805 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 295) took 3329 ms. Trace:
I20260808 13:35:35.493189 7014 rpcz_store.cc:276] 0808 13:35:32.163646 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.165569 (+ 1923us) service_pool.cc:227] Handling call
0808 13:35:35.492346 (+3326777us) ts_tablet_manager.cc:1145] Deleting tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be
0808 13:35:35.492402 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.492731 (+ 329us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3326177}
W20260808 13:35:35.494688 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be already in progress: creating tablet
I20260808 13:35:35.502310 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 52e8243692d847039aafda3e0d0095fb (DEFAULT_TABLE table=default.Table_01763d5cef124bad885eaaba0686787b [id=8269e8c2637f41859e8ead761bc2d7bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:35.506021 6046 tablet_bootstrap.cc:492] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:35.506791 6046 ts_tablet_manager.cc:1397] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.227s user 0.019s sys 0.000s
I20260808 13:35:35.522362 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264ee45f8dfa4a05b0e5e812ce1b622c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:35.522758 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 296) took 3180 ms. Trace:
I20260808 13:35:35.522922 5882 raft_consensus.cc:3037] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:35.523058 7014 rpcz_store.cc:276] 0808 13:35:32.342567 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.342711 (+ 144us) service_pool.cc:227] Handling call
0808 13:35:35.522361 (+3179650us) ts_tablet_manager.cc:1145] Deleting tablet 20113491b108476b95b8f32b69176d0e
0808 13:35:35.522408 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.522687 (+ 279us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3178966}
W20260808 13:35:35.524418 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 20113491b108476b95b8f32b69176d0e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 20113491b108476b95b8f32b69176d0e already in progress: creating tablet
I20260808 13:35:35.524642 4835 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_953730643273433c8353604101a4d237 [id=d9baa62ea5c249a898586a45f8b4f372] alter complete (version 2)
I20260808 13:35:35.528570 5832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0767140736242f18ef4689b112deefb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.533188 6046 raft_consensus.cc:348] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.533981 6046 raft_consensus.cc:374] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:35.534257 6046 raft_consensus.cc:729] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:35.490370 7199 ts_tablet_manager.cc:1918] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:35.535405 7199 log.cc:1196] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/31fc9fdb5ef543da833f9af6ca5807a9
I20260808 13:35:35.536453 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 297) took 2301 ms. Trace:
I20260808 13:35:35.502556 7013 ts_tablet_manager.cc:1918] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260808 13:35:35.537292 4836 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 538d9a883e9d46668fde30876352d20e consensus state has no leader
I20260808 13:35:35.538901 7199 ts_tablet_manager.cc:1939] T 31fc9fdb5ef543da833f9af6ca5807a9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:35.539328 7013 log.cc:1196] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/d3682c9c656c4556bc13ae3aeddd7c5e
I20260808 13:35:35.540478 7013 ts_tablet_manager.cc:1939] T d3682c9c656c4556bc13ae3aeddd7c5e P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:35.540936 7014 rpcz_store.cc:276] 0808 13:35:33.235357 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.235521 (+ 164us) service_pool.cc:227] Handling call
0808 13:35:35.536061 (+2300540us) ts_tablet_manager.cc:1145] Deleting tablet 3451d72548a140c68a6ccc2b566a09b1
0808 13:35:35.536110 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.536386 (+ 276us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2299903}
I20260808 13:35:35.545332 6046 consensus_queue.cc:260] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:35.547528 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 3451d72548a140c68a6ccc2b566a09b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3451d72548a140c68a6ccc2b566a09b1 already in progress: creating tablet
I20260808 13:35:35.549660 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f2e0b98af431d99894e8778257bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:35.552032 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddd0134819474e7eadd4e3d12d94a7b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.552418 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 298) took 2315 ms. Trace:
I20260808 13:35:35.552721 7014 rpcz_store.cc:276] 0808 13:35:33.236571 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.236721 (+ 150us) service_pool.cc:227] Handling call
0808 13:35:35.552066 (+2315345us) ts_tablet_manager.cc:1145] Deleting tablet 1ba5fa465ec04bbb801a821a71a8f87d
0808 13:35:35.552111 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.552362 (+ 251us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2314723}
W20260808 13:35:35.553932 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 1ba5fa465ec04bbb801a821a71a8f87d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ba5fa465ec04bbb801a821a71a8f87d already in progress: creating tablet
I20260808 13:35:35.554594 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:35.555140 5826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 205532710f0148f38d12a485bc0107ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.557168 5834 tablet_service.cc:1514] Processing DeleteTablet for tablet 69642fdba1f145438b04eae9f8154101 with delete_type TABLET_DATA_DELETED (replaced by tablet 538d9a883e9d46668fde30876352d20e) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:35.558393 4841 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet d9268f5f6dee4112932e106010a3c2be consensus state has no leader
W20260808 13:35:35.561234 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 36) took 3101 ms (client timeout 1000 ms). Trace:
W20260808 13:35:35.561550 4832 rpcz_store.cc:269] 0808 13:35:32.459430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.459563 (+ 133us) service_pool.cc:227] Handling call
0808 13:35:35.561147 (+3101584us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16392,"reported_tablets":20}
I20260808 13:35:35.562179 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2483b3320e2e49db96d39254dd5755aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.564142 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 302) took 2259 ms. Trace:
I20260808 13:35:35.564457 7014 rpcz_store.cc:276] 0808 13:35:33.304540 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.304945 (+ 405us) service_pool.cc:227] Handling call
0808 13:35:35.563764 (+2258819us) ts_tablet_manager.cc:1145] Deleting tablet 61dfd47db0a74ed8b7228b559dd68459
0808 13:35:35.563808 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.564082 (+ 274us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2258233}
W20260808 13:35:35.560824 4841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 37) took 1781 ms (client timeout 1000 ms). Trace:
W20260808 13:35:35.565997 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 61dfd47db0a74ed8b7228b559dd68459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 61dfd47db0a74ed8b7228b559dd68459 already in progress: creating tablet
I20260808 13:35:35.567104 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e8243692d847039aafda3e0d0095fb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:35.567586 4838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 45) took 3465 ms (client timeout 1000 ms). Trace:
I20260808 13:35:35.568220 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:35.568216 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9bc338093f403881d329a7e913c9f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:20 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:35.568897 4841 rpcz_store.cc:269] 0808 13:35:33.778984 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.799050 (+ 20066us) service_pool.cc:227] Handling call
0808 13:35:33.836282 (+ 37232us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:35.560760 (+1724478us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":32226,"reported_tablets":26,"spinlock_wait_cycles":65154304}
I20260808 13:35:35.570960 6046 ts_tablet_manager.cc:1428] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.064s user 0.009s sys 0.000s
I20260808 13:35:35.571508 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809049a771224082a16c270d01ec4bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.572752 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:25 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:35.573589 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 127e8d0323954ef788018bcb91582951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:35.575193 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2330be58e9d6414da65b4434c2d3b461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:35.576911 7568 raft_consensus.cc:482] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:35.577688 7568 raft_consensus.cc:504] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.579700 6033 tablet_bootstrap.cc:492] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:35.579957 7568 leader_election.cc:288] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:35.583938 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1847d0fecb843cdbc833ad674a503a4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:35.584460 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:35.585191 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet 896389941179456d9d0ed3cd14726f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:35.588241 6046 tablet_bootstrap.cc:492] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:35.590188 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e18e51c2e24d3caa3d29e6645e46ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:35.591279 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 314) took 2169 ms. Trace:
I20260808 13:35:35.592453 7014 rpcz_store.cc:276] 0808 13:35:33.422047 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.426903 (+ 4856us) service_pool.cc:227] Handling call
0808 13:35:35.590942 (+2164039us) ts_tablet_manager.cc:1145] Deleting tablet cb7f6487e28f4aaa94d448be71f8a3c2
0808 13:35:35.590984 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.591223 (+ 239us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2163307}
W20260808 13:35:35.593670 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet cb7f6487e28f4aaa94d448be71f8a3c2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb7f6487e28f4aaa94d448be71f8a3c2 already in progress: creating tablet
I20260808 13:35:35.597144 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 9423b380a62c467895e7fdff14756586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:35.599231 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 281) took 5775 ms. Trace:
I20260808 13:35:35.599681 7199 rpcz_store.cc:276] 0808 13:35:29.823988 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:29.824457 (+ 469us) service_pool.cc:227] Handling call
0808 13:35:35.132141 (+5307684us) ts_tablet_manager.cc:1145] Deleting tablet 31fc9fdb5ef543da833f9af6ca5807a9
0808 13:35:35.132220 (+ 79us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.455050 (+322830us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:35.490138 (+ 35088us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:35.599175 (+109037us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5306962}
I20260808 13:35:35.600575 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 291) took 5349 ms. Trace:
I20260808 13:35:35.602391 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 31fc9fdb5ef543da833f9af6ca5807a9 (table default.Table_faaabef2a0ab42648427c33cda679fb7 [id=7afc114b9ab641df8c540f834036b70c]) successfully deleted
I20260808 13:35:35.603411 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 31fc9fdb5ef543da833f9af6ca5807a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 67327badb2524a2c817045a893bcad6e) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:35.603570 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 286) took 5253 ms. Trace:
I20260808 13:35:35.603852 7199 rpcz_store.cc:276] 0808 13:35:30.349886 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.350030 (+ 144us) service_pool.cc:227] Handling call
0808 13:35:35.603140 (+5253110us) ts_tablet_manager.cc:1145] Deleting tablet ba39706cff65404485959865fff1772a
0808 13:35:35.603194 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.603521 (+ 327us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5252597}
I20260808 13:35:35.603886 7013 rpcz_store.cc:276] 0808 13:35:30.251471 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.251589 (+ 118us) service_pool.cc:227] Handling call
0808 13:35:35.146770 (+4895181us) ts_tablet_manager.cc:1145] Deleting tablet d3682c9c656c4556bc13ae3aeddd7c5e
0808 13:35:35.248345 (+101575us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.353889 (+105544us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:35.502341 (+148452us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:35.600512 (+ 98171us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":32737,"tablet-delete.queue_time_us":4894531}
W20260808 13:35:35.604863 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet ba39706cff65404485959865fff1772a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba39706cff65404485959865fff1772a already in progress: creating tablet
I20260808 13:35:35.606312 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 287) took 5084 ms. Trace:
I20260808 13:35:35.607653 7014 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 317) took 1792 ms. Trace:
I20260808 13:35:35.607960 7014 rpcz_store.cc:276] 0808 13:35:33.815383 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.833222 (+ 17839us) service_pool.cc:227] Handling call
0808 13:35:35.607330 (+1774108us) ts_tablet_manager.cc:1145] Deleting tablet 293e9540498e420d8ec873472d0d29fc
0808 13:35:35.607370 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.607597 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1758266}
W20260808 13:35:35.609054 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 293e9540498e420d8ec873472d0d29fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 293e9540498e420d8ec873472d0d29fc already in progress: creating tablet
I20260808 13:35:35.606648 7199 rpcz_store.cc:276] 0808 13:35:30.521672 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.521805 (+ 133us) service_pool.cc:227] Handling call
0808 13:35:35.605519 (+5083714us) ts_tablet_manager.cc:1145] Deleting tablet 9ea8f5322a404837b91e1d96749926b7
0808 13:35:35.605792 (+ 273us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.606265 (+ 473us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5082984}
I20260808 13:35:35.608502 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 292) took 5354 ms. Trace:
W20260808 13:35:35.568027 4838 rpcz_store.cc:269] 0808 13:35:32.102351 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.113097 (+ 10746us) service_pool.cc:227] Handling call
0808 13:35:32.173496 (+ 60399us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:35.567523 (+3394027us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:33.477099 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:33.495139 (+ 18040us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:33.495164 (+ 25us) write_op.cc:435] Acquired schema lock
0808 13:35:33.495179 (+ 15us) tablet.cc:660] Decoding operations
0808 13:35:33.495472 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:33.495513 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:33.495535 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:33.495682 (+ 147us) tablet.cc:699] Row locks acquired
0808 13:35:33.495696 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:33.495810 (+ 114us) write_op.cc:270] Start()
0808 13:35:33.495895 (+ 85us) write_op.cc:276] Timestamp: P: 1786196133495761 usec, L: 0
0808 13:35:33.495912 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:33.496448 (+ 536us) log.cc:842] Serialized 356 byte log entry
0808 13:35:33.603035 (+106587us) op_driver.cc:444] REPLICATION: finished
0808 13:35:33.605195 (+ 2160us) write_op.cc:301] APPLY: starting
0808 13:35:33.605252 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:33.605341 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:33.605479 (+ 138us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:33.605686 (+ 207us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:33.605831 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:33.605892 (+ 61us) write_op.cc:312] APPLY: finished
0808 13:35:33.606072 (+ 180us) log.cc:842] Serialized 23 byte log entry
0808 13:35:33.606214 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:33.606270 (+ 56us) write_op.cc:454] Released schema lock
0808 13:35:33.606298 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:33.606460 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:33.606486 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14045,"reported_tablets":13,"spinlock_wait_cycles":92329856,"child_traces":[["op",{"apply.queue_time_us":1990,"apply.run_cpu_time_us":1433,"apply.run_wall_time_us":1433,"num_ops":1,"prepare.queue_time_us":175499,"prepare.run_cpu_time_us":2056,"prepare.run_wall_time_us":22003,"raft.queue_time_us":90,"raft.run_cpu_time_us":722,"raft.run_wall_time_us":721,"replication_time_us":107012}]]}
I20260808 13:35:35.611666 7013 rpcz_store.cc:276] 0808 13:35:30.254426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.254542 (+ 116us) service_pool.cc:227] Handling call
0808 13:35:35.608201 (+5353659us) ts_tablet_manager.cc:1145] Deleting tablet f0c76b819474468baac51365360becfc
0808 13:35:35.608254 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.608446 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5352442}
I20260808 13:35:35.614741 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 69642fdba1f145438b04eae9f8154101 with delete_type TABLET_DATA_DELETED (replaced by tablet 538d9a883e9d46668fde30876352d20e) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:35.616560 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 293) took 4251 ms. Trace:
I20260808 13:35:35.616866 7199 rpcz_store.cc:276] 0808 13:35:31.364944 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:31.365098 (+ 154us) service_pool.cc:227] Handling call
0808 13:35:35.615995 (+4250897us) ts_tablet_manager.cc:1145] Deleting tablet 3272708a835c432c878cc330e44490c5
0808 13:35:35.616099 (+ 104us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.616503 (+ 404us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4250284}
I20260808 13:35:35.621850 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 293) took 5365 ms. Trace:
I20260808 13:35:35.623674 7013 rpcz_store.cc:276] 0808 13:35:30.256249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.256363 (+ 114us) service_pool.cc:227] Handling call
0808 13:35:35.621355 (+5364992us) ts_tablet_manager.cc:1145] Deleting tablet 9260370cf8cb46a7b699581e4e8d46db
0808 13:35:35.621411 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.621790 (+ 379us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5363980}
I20260808 13:35:35.625341 7014 tablet_replica.cc:330] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:35.627213 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 296) took 4039 ms. Trace:
I20260808 13:35:35.635232 7199 rpcz_store.cc:276] 0808 13:35:31.587422 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:31.603551 (+ 16129us) service_pool.cc:227] Handling call
0808 13:35:35.626663 (+4023112us) ts_tablet_manager.cc:1145] Deleting tablet 62d68ab73bae4dd9af46e907b1fd198c
0808 13:35:35.626733 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.627127 (+ 394us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4020636}
I20260808 13:35:35.639014 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 302) took 3855 ms. Trace:
W20260808 13:35:35.611928 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 9ea8f5322a404837b91e1d96749926b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ea8f5322a404837b91e1d96749926b7 already in progress: creating tablet
I20260808 13:35:35.642849 7199 rpcz_store.cc:276] 0808 13:35:31.783606 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:31.784019 (+ 413us) service_pool.cc:227] Handling call
0808 13:35:35.638709 (+3854690us) ts_tablet_manager.cc:1145] Deleting tablet d3682c9c656c4556bc13ae3aeddd7c5e
0808 13:35:35.638772 (+ 63us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.638966 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3854039}
W20260808 13:35:35.643412 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 3272708a835c432c878cc330e44490c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3272708a835c432c878cc330e44490c5 already in progress: creating tablet
W20260808 13:35:35.644034 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 62d68ab73bae4dd9af46e907b1fd198c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62d68ab73bae4dd9af46e907b1fd198c already in progress: creating tablet
W20260808 13:35:35.644618 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet d3682c9c656c4556bc13ae3aeddd7c5e because the tablet was not found. No further retry: Not found: Tablet not found: d3682c9c656c4556bc13ae3aeddd7c5e
W20260808 13:35:35.612998 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet f0c76b819474468baac51365360becfc because the tablet was not found. No further retry: Not found: Tablet not found: f0c76b819474468baac51365360becfc
I20260808 13:35:35.646332 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 303) took 3862 ms. Trace:
I20260808 13:35:35.649376 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_82d49330ca914820a6cb8b3591280a90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:35.655145 7199 rpcz_store.cc:276] 0808 13:35:31.783892 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:31.784015 (+ 123us) service_pool.cc:227] Handling call
0808 13:35:35.645981 (+3861966us) ts_tablet_manager.cc:1145] Deleting tablet e992a0294c404cfb9541b57782a05fc9
0808 13:35:35.646046 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.646274 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3861217}
I20260808 13:35:35.667075 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 295) took 5395 ms. Trace:
I20260808 13:35:35.667937 7013 rpcz_store.cc:276] 0808 13:35:30.271739 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.271877 (+ 138us) service_pool.cc:227] Handling call
0808 13:35:35.666530 (+5394653us) ts_tablet_manager.cc:1145] Deleting tablet 2cd2d0c9ab2c4704a74d06559bb913d6
0808 13:35:35.666641 (+ 111us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.666997 (+ 356us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5393508}
I20260808 13:35:35.647224 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e with delete_type TABLET_DATA_DELETED (replaced by tablet ba39706cff65404485959865fff1772a) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:35.663097 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet 31fc9fdb5ef543da833f9af6ca5807a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 67327badb2524a2c817045a893bcad6e) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:35.671020 5551 leader_election.cc:341] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:35.646973 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet ddfd9e3f002445738df24ebb0a333440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:35.681787 6046 tablet_bootstrap.cc:654] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:35.685940 6026 tablet_bootstrap.cc:654] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:35.693467 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 296) took 5341 ms. Trace:
I20260808 13:35:35.693922 7013 rpcz_store.cc:276] 0808 13:35:30.351921 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.352084 (+ 163us) service_pool.cc:227] Handling call
0808 13:35:35.692805 (+5340721us) ts_tablet_manager.cc:1145] Deleting tablet ba39706cff65404485959865fff1772a
0808 13:35:35.692903 (+ 98us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.693396 (+ 493us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5339990}
I20260808 13:35:35.695521 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 297) took 5182 ms. Trace:
I20260808 13:35:35.698273 7013 rpcz_store.cc:276] 0808 13:35:30.513152 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:30.644720 (+131568us) service_pool.cc:227] Handling call
0808 13:35:35.695028 (+5050308us) ts_tablet_manager.cc:1145] Deleting tablet 9ea8f5322a404837b91e1d96749926b7
0808 13:35:35.695134 (+ 106us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.695474 (+ 340us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5024207}
I20260808 13:35:35.706523 7434 consensus_queue.cc:1035] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:35.706921 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 303) took 4109 ms. Trace:
I20260808 13:35:35.709613 7013 rpcz_store.cc:276] 0808 13:35:31.597437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:31.597559 (+ 122us) service_pool.cc:227] Handling call
0808 13:35:35.705025 (+4107466us) ts_tablet_manager.cc:1145] Deleting tablet 3272708a835c432c878cc330e44490c5
0808 13:35:35.705159 (+ 134us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.706832 (+ 1673us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4085881}
I20260808 13:35:35.711508 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 306) took 4113 ms. Trace:
W20260808 13:35:35.619459 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 1f0a6d9bd2c74d0790badd7977f027e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f0a6d9bd2c74d0790badd7977f027e3 already in progress: creating tablet
I20260808 13:35:35.712954 7013 rpcz_store.cc:276] 0808 13:35:31.598335 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:31.598697 (+ 362us) service_pool.cc:227] Handling call
0808 13:35:35.710844 (+4112147us) ts_tablet_manager.cc:1145] Deleting tablet 62d68ab73bae4dd9af46e907b1fd198c
0808 13:35:35.710945 (+ 101us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.711431 (+ 486us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4088845}
W20260808 13:35:35.713129 4821 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 93690.5us
W20260808 13:35:35.713845 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82d49330ca914820a6cb8b3591280a90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:35.714395 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 6e70a5cf88d94b36913a6d1ccfcb979a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e70a5cf88d94b36913a6d1ccfcb979a already in progress: creating tablet
W20260808 13:35:35.623652 4842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 47) took 1250 ms (client timeout 1000 ms). Trace:
I20260808 13:35:35.626647 4836 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 365) took 3080 ms. Trace:
I20260808 13:35:35.718047 7014 raft_consensus.cc:2229] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:35.718572 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 311) took 3686 ms. Trace:
I20260808 13:35:35.718765 7014 raft_consensus.cc:2258] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:35.718930 7013 rpcz_store.cc:276] 0808 13:35:32.031841 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.032057 (+ 216us) service_pool.cc:227] Handling call
0808 13:35:35.718176 (+3686119us) ts_tablet_manager.cc:1145] Deleting tablet 4625b632d67244fc86e48d739dde707e
0808 13:35:35.718256 (+ 80us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.718521 (+ 265us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3684527}
I20260808 13:35:35.720847 7014 ts_tablet_manager.cc:1905] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:35.723913 7013 tablet_replica.cc:330] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
W20260808 13:35:35.729583 4839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 51) took 3927 ms (client timeout 1000 ms). Trace:
W20260808 13:35:35.609077 4837 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 46) took 2330 ms (client timeout 1000 ms). Trace:
W20260808 13:35:35.666181 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 9260370cf8cb46a7b699581e4e8d46db because the tablet was not found. No further retry: Not found: Tablet not found: 9260370cf8cb46a7b699581e4e8d46db
I20260808 13:35:35.730630 7013 raft_consensus.cc:2229] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
W20260808 13:35:35.731022 4820 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 64860.3us
I20260808 13:35:35.731635 7013 raft_consensus.cc:2258] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:35.733848 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 2cd2d0c9ab2c4704a74d06559bb913d6 because the tablet was not found. No further retry: Not found: Tablet not found: 2cd2d0c9ab2c4704a74d06559bb913d6
I20260808 13:35:35.734035 7013 ts_tablet_manager.cc:1905] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:35.667369 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet e992a0294c404cfb9541b57782a05fc9 because the tablet was not found. No further retry: Not found: Tablet not found: e992a0294c404cfb9541b57782a05fc9
W20260808 13:35:35.734792 4819 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 67463.3us
I20260808 13:35:35.738137 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:25 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:35.739781 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet 127e8d0323954ef788018bcb91582951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:35.740334 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 with delete_type TABLET_DATA_DELETED (replaced by tablet d8b07b7d31ef41e484389c0f007598f0) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:35.740847 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e18e51c2e24d3caa3d29e6645e46ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:35.741433 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 69642fdba1f145438b04eae9f8154101 with delete_type TABLET_DATA_DELETED (replaced by tablet 538d9a883e9d46668fde30876352d20e) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:35.742264 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9bc338093f403881d329a7e913c9f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:20 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:35.734503 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet ba39706cff65404485959865fff1772a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba39706cff65404485959865fff1772a already in progress: creating tablet
W20260808 13:35:35.744115 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 9ea8f5322a404837b91e1d96749926b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ea8f5322a404837b91e1d96749926b7 already in progress: creating tablet
W20260808 13:35:35.744854 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 3272708a835c432c878cc330e44490c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3272708a835c432c878cc330e44490c5 already in progress: creating tablet
W20260808 13:35:35.745620 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 62d68ab73bae4dd9af46e907b1fd198c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62d68ab73bae4dd9af46e907b1fd198c already in progress: creating tablet
W20260808 13:35:35.746323 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 4625b632d67244fc86e48d739dde707e because the tablet was not found. No further retry: Not found: Tablet not found: 4625b632d67244fc86e48d739dde707e
W20260808 13:35:35.742827 4837 rpcz_store.cc:269] 0808 13:35:33.278573 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.300715 (+ 22142us) service_pool.cc:227] Handling call
0808 13:35:33.344503 (+ 43788us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:35.609007 (+2264504us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4626,"reported_tablets":19,"spinlock_wait_cycles":50869504}
I20260808 13:35:35.748812 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a8ebe3d6290044f79864293e5c170dd5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:35.750684 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8ebe3d6290044f79864293e5c170dd5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:35.628568 5882 raft_consensus.cc:2454] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:35.752997 5550 leader_election.cc:302] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:35.723237 4836 rpcz_store.cc:276] 0808 13:35:32.545739 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.545849 (+ 110us) service_pool.cc:227] Handling call
0808 13:35:32.607324 (+ 61475us) spinlock_profiling.cc:243] Waited 55.9 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff7d6 00007f227bd45634 00007f227bd455c2 00007f227bd454f1 00007f227bd4546c 00007f227bd44722 00007f2279fde94c 00007f2279fddd86
0808 13:35:32.607979 (+ 655us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:32.608593 (+ 614us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:32.609151 (+ 558us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:32.609330 (+ 179us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:32.655233 (+ 45903us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:35.626546 (+2971313us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:32.623982 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:32.624310 (+ 328us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:32.624331 (+ 21us) write_op.cc:435] Acquired schema lock
0808 13:35:32.624342 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:32.624588 (+ 246us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:32.624624 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:32.624646 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:32.624786 (+ 140us) tablet.cc:699] Row locks acquired
0808 13:35:32.624799 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:32.624896 (+ 97us) write_op.cc:270] Start()
0808 13:35:32.624974 (+ 78us) write_op.cc:276] Timestamp: P: 1786196132624828 usec, L: 0
0808 13:35:32.624990 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:32.625453 (+ 463us) log.cc:842] Serialized 405 byte log entry
0808 13:35:32.652996 (+ 27543us) op_driver.cc:444] REPLICATION: finished
0808 13:35:32.653211 (+ 215us) write_op.cc:301] APPLY: starting
0808 13:35:32.653261 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:32.653352 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:32.653460 (+ 108us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:32.653624 (+ 164us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:32.653741 (+ 117us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:32.653787 (+ 46us) write_op.cc:312] APPLY: finished
0808 13:35:32.653974 (+ 187us) log.cc:842] Serialized 23 byte log entry
0808 13:35:32.654111 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:32.654181 (+ 70us) write_op.cc:454] Released schema lock
0808 13:35:32.654214 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:32.654396 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:32.654419 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":128609024,"child_traces":[["op",{"apply.queue_time_us":76,"apply.run_cpu_time_us":1351,"apply.run_wall_time_us":1351,"num_ops":1,"prepare.queue_time_us":13498,"prepare.run_cpu_time_us":1744,"prepare.run_wall_time_us":22002,"raft.queue_time_us":1290,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":27929}]]}
I20260808 13:35:35.757025 7690 raft_consensus.cc:2781] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:35.761004 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_8a69b0e5577e4c6f9c76504751d2c1a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:35.763199 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a69b0e5577e4c6f9c76504751d2c1a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:35.771601 7690 raft_consensus.cc:686] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:35.774048 6046 tablet_bootstrap.cc:492] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:35.774506 6046 ts_tablet_manager.cc:1397] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.188s user 0.020s sys 0.003s
I20260808 13:35:35.785346 7664 tablet.cc:1722] T c4c083c8e19d4279af600670f5d1d78d P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260808 13:35:35.797773 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 520415us
W20260808 13:35:35.773245 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 52) took 2658 ms (client timeout 1000 ms). Trace:
I20260808 13:35:35.806432 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0cc688386f3413ea939dda3a75d6541" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:35.807448 5640 raft_consensus.cc:3037] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:35.729918 4839 rpcz_store.cc:269] 0808 13:35:31.802506 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:31.816740 (+ 14234us) service_pool.cc:227] Handling call
0808 13:35:31.992958 (+176218us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:35.729467 (+3736509us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":884,"reported_tablets":27,"spinlock_wait_cycles":393490816}
W20260808 13:35:35.715938 4842 rpcz_store.cc:269] 0808 13:35:34.373322 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.828410 (+455088us) service_pool.cc:227] Handling call
0808 13:35:34.915954 (+ 87544us) spinlock_profiling.cc:243] Waited 74.7 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:35.623577 (+707623us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:35.356600 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:35.356953 (+ 353us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:35.356975 (+ 22us) write_op.cc:435] Acquired schema lock
0808 13:35:35.356988 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:35.357248 (+ 260us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:35.357287 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:35.357309 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:35.357461 (+ 152us) tablet.cc:699] Row locks acquired
0808 13:35:35.357474 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:35.357576 (+ 102us) write_op.cc:270] Start()
0808 13:35:35.357652 (+ 76us) write_op.cc:276] Timestamp: P: 1786196135357497 usec, L: 0
0808 13:35:35.357667 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:35.358125 (+ 458us) log.cc:842] Serialized 527 byte log entry
0808 13:35:35.371330 (+ 13205us) op_driver.cc:444] REPLICATION: finished
0808 13:35:35.371806 (+ 476us) write_op.cc:301] APPLY: starting
0808 13:35:35.371865 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:35.371984 (+ 119us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:35.372137 (+ 153us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:35.372469 (+ 332us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:35.372760 (+ 291us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:35.372816 (+ 56us) write_op.cc:312] APPLY: finished
0808 13:35:35.373158 (+ 342us) log.cc:842] Serialized 23 byte log entry
0808 13:35:35.373383 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.373474 (+ 91us) write_op.cc:454] Released schema lock
0808 13:35:35.373567 (+ 93us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:35.373836 (+ 269us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.373858 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":132977,"reported_tablets":20,"spinlock_wait_cycles":171872512,"child_traces":[["op",{"apply.queue_time_us":253,"apply.run_cpu_time_us":2283,"apply.run_wall_time_us":2283,"num_ops":1,"prepare.queue_time_us":95,"prepare.run_cpu_time_us":1794,"prepare.run_wall_time_us":1795,"raft.queue_time_us":1158,"raft.run_cpu_time_us":639,"raft.run_wall_time_us":639,"replication_time_us":13548}]]}
I20260808 13:35:35.813882 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b5830adfb9064ab4b32c29c4c1a03066" } new_table_name: "default.Table_6eaa73a8f6204e05808ac7d1b397ec04" modify_external_catalogs: true
I20260808 13:35:35.816243 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 307) took 3656 ms. Trace:
I20260808 13:35:35.816748 7199 rpcz_store.cc:276] 0808 13:35:32.160265 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.192948 (+ 32683us) service_pool.cc:227] Handling call
0808 13:35:35.763805 (+3570857us) ts_tablet_manager.cc:1145] Deleting tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be
0808 13:35:35.763857 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.816186 (+ 52329us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3569721}
I20260808 13:35:35.773475 7690 consensus_queue.cc:237] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:35.818481 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be already in progress: creating tablet
I20260808 13:35:35.833372 6033 tablet_bootstrap.cc:654] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:35.797624 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0cc688386f3413ea939dda3a75d6541" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:35.778445 6046 raft_consensus.cc:348] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.839147 6046 raft_consensus.cc:374] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:35.839519 6046 raft_consensus.cc:729] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:35.841359 7718 tablet.cc:1722] T c4c083c8e19d4279af600670f5d1d78d P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:35.847324 4839 master_service.cc:946] ReplaceTablet: received request to replace tablet 56d8d5a3ca724141b58e4e2cea23be68 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:35.831382 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba098c946794607b2a15bc64098d918" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:35.849869 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 306) took 3690 ms. Trace:
I20260808 13:35:35.850445 7199 rpcz_store.cc:276] 0808 13:35:32.159838 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.195097 (+ 35259us) service_pool.cc:227] Handling call
0808 13:35:35.818127 (+3623030us) ts_tablet_manager.cc:1145] Deleting tablet 8d12106f0d794a79935999a2b8dbe7e9
0808 13:35:35.818169 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.849795 (+ 31626us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3622341}
W20260808 13:35:35.852150 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 8d12106f0d794a79935999a2b8dbe7e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d12106f0d794a79935999a2b8dbe7e9 already in progress: creating tablet
W20260808 13:35:35.860731 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b6b421ff1f094d8689010b4ed6c3f7af consensus state has no leader
I20260808 13:35:35.861764 4842 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 396) took 1452 ms. Trace:
I20260808 13:35:35.833990 7702 raft_consensus.cc:482] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:35.806857 4835 rpcz_store.cc:269] 0808 13:35:33.115058 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.150326 (+ 35268us) service_pool.cc:227] Handling call
0808 13:35:35.773161 (+2622835us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:33.830010 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:33.830427 (+ 417us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:33.830454 (+ 27us) write_op.cc:435] Acquired schema lock
0808 13:35:33.830472 (+ 18us) tablet.cc:660] Decoding operations
0808 13:35:33.830812 (+ 340us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:33.830860 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:33.830887 (+ 27us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:33.831059 (+ 172us) tablet.cc:699] Row locks acquired
0808 13:35:33.831081 (+ 22us) write_op.cc:260] PREPARE: finished
0808 13:35:33.831199 (+ 118us) write_op.cc:270] Start()
0808 13:35:33.831293 (+ 94us) write_op.cc:276] Timestamp: P: 1786196133831115 usec, L: 0
0808 13:35:33.831312 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:33.831895 (+ 583us) log.cc:842] Serialized 527 byte log entry
0808 13:35:33.872962 (+ 41067us) op_driver.cc:444] REPLICATION: finished
0808 13:35:33.880665 (+ 7703us) write_op.cc:301] APPLY: starting
0808 13:35:33.880717 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:33.880804 (+ 87us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:33.880904 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:33.881102 (+ 198us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:33.881218 (+ 116us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:33.881256 (+ 38us) write_op.cc:312] APPLY: finished
0808 13:35:33.881412 (+ 156us) log.cc:842] Serialized 23 byte log entry
0808 13:35:33.884572 (+ 3160us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:33.884631 (+ 59us) write_op.cc:454] Released schema lock
0808 13:35:33.884668 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:33.884850 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:33.884875 (+ 25us) write_op.cc:454] Released schema lock
Related trace 'op':
0808 13:35:34.966706 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:34.967171 (+ 465us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:34.967196 (+ 25us) write_op.cc:435] Acquired schema lock
0808 13:35:34.967234 (+ 38us) tablet.cc:660] Decoding operations
0808 13:35:34.967545 (+ 311us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:34.967591 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:34.967614 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:34.967804 (+ 190us) tablet.cc:699] Row locks acquired
0808 13:35:34.967819 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:34.967928 (+ 109us) write_op.cc:270] Start()
0808 13:35:34.968014 (+ 86us) write_op.cc:276] Timestamp: P: 1786196134967889 usec, L: 0
0808 13:35:34.968030 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:34.968673 (+ 643us) log.cc:842] Serialized 356 byte log entry
0808 13:35:34.988869 (+ 20196us) op_driver.cc:444] REPLICATION: finished
0808 13:35:34.993540 (+ 4671us) write_op.cc:301] APPLY: starting
0808 13:35:34.993643 (+ 103us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:34.993955 (+ 312us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:34.994153 (+ 198us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:34.994482 (+ 329us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:34.994766 (+ 284us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:34.994879 (+ 113us) write_op.cc:312] APPLY: finished
0808 13:35:34.995195 (+ 316us) log.cc:842] Serialized 23 byte log entry
0808 13:35:34.995883 (+ 688us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:34.996016 (+ 133us) write_op.cc:454] Released schema lock
0808 13:35:34.996075 (+ 59us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:34.997521 (+ 1446us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:34.997611 (+ 90us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":7546,"apply.run_cpu_time_us":1381,"apply.run_wall_time_us":4368,"num_ops":1,"prepare.queue_time_us":403,"prepare.run_cpu_time_us":2214,"prepare.run_wall_time_us":3415,"raft.queue_time_us":633,"raft.run_cpu_time_us":17,"raft.run_wall_time_us":26,"replication_time_us":41572}],["op",{"apply.queue_time_us":4404,"apply.run_cpu_time_us":2851,"apply.run_wall_time_us":4321,"num_ops":1,"prepare.queue_time_us":109,"prepare.run_cpu_time_us":2412,"prepare.run_wall_time_us":2419,"raft.queue_time_us":4841,"raft.run_cpu_time_us":697,"raft.run_wall_time_us":13391,"replication_time_us":20696}]]}
I20260808 13:35:35.864454 4842 rpcz_store.cc:276] 0808 13:35:34.408950 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.813541 (+1404591us) service_pool.cc:227] Handling call
0808 13:35:35.821983 (+ 8442us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:35.822992 (+ 1009us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:35.823832 (+ 840us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:35.824116 (+ 284us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:35.855071 (+ 30955us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.855111 (+ 40us) write_op.cc:454] Released schema lock
0808 13:35:35.855365 (+ 254us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:35.861684 (+ 6319us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:35.825697 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:35.826138 (+ 441us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:35.826170 (+ 32us) write_op.cc:435] Acquired schema lock
0808 13:35:35.826189 (+ 19us) tablet.cc:660] Decoding operations
0808 13:35:35.826489 (+ 300us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:35.826637 (+ 148us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:35.826662 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:35.826818 (+ 156us) tablet.cc:699] Row locks acquired
0808 13:35:35.826831 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:35.826969 (+ 138us) write_op.cc:270] Start()
0808 13:35:35.827111 (+ 142us) write_op.cc:276] Timestamp: P: 1786196135826840 usec, L: 0
0808 13:35:35.827127 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:35.835758 (+ 8631us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b5400021fa0. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c1c4dc0 00007f227c235bd2 00007f227c22b2d7 00007f227c22b114 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1
0808 13:35:35.836262 (+ 504us) log.cc:842] Serialized 405 byte log entry
0808 13:35:35.848087 (+ 11825us) op_driver.cc:444] REPLICATION: finished
0808 13:35:35.849714 (+ 1627us) write_op.cc:301] APPLY: starting
0808 13:35:35.849906 (+ 192us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:35.850067 (+ 161us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:35.850108 (+ 41us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:35.850594 (+ 486us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:35.850873 (+ 279us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:35.850953 (+ 80us) write_op.cc:312] APPLY: finished
0808 13:35:35.851315 (+ 362us) log.cc:842] Serialized 23 byte log entry
0808 13:35:35.851596 (+ 281us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.851703 (+ 107us) write_op.cc:454] Released schema lock
0808 13:35:35.851751 (+ 48us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1370,"apply.run_cpu_time_us":2316,"apply.run_wall_time_us":2400,"num_ops":1,"prepare.queue_time_us":109,"prepare.run_cpu_time_us":5083,"prepare.run_wall_time_us":13326,"raft.queue_time_us":87,"raft.run_cpu_time_us":1270,"raft.run_wall_time_us":1278,"replication_time_us":20856,"spinlock_wait_cycles":13267712}]]}
I20260808 13:35:35.887775 4835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_2a111b21bed74a8bb3c9822dff0c2cc6" } modify_external_catalogs: true
I20260808 13:35:35.890538 4835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_2a111b21bed74a8bb3c9822dff0c2cc6" } modify_external_catalogs: true
I20260808 13:35:35.852623 6046 consensus_queue.cc:260] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.893195 4836 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 395) took 1525 ms. Trace:
I20260808 13:35:35.893592 6046 ts_tablet_manager.cc:1428] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.119s user 0.003s sys 0.004s
I20260808 13:35:35.893553 4836 rpcz_store.cc:276] 0808 13:35:34.368327 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.760429 (+1392102us) service_pool.cc:227] Handling call
0808 13:35:35.794813 (+ 34384us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:35.796547 (+ 1734us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:35.892572 (+ 96025us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:35.892889 (+ 317us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:35.893142 (+ 253us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:35.855513 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:35.856110 (+ 597us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:35.856134 (+ 24us) write_op.cc:435] Acquired schema lock
0808 13:35:35.856147 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:35.856493 (+ 346us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:35.856535 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:35.856556 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:35.856764 (+ 208us) tablet.cc:699] Row locks acquired
0808 13:35:35.874077 (+ 17313us) write_op.cc:260] PREPARE: finished
0808 13:35:35.874232 (+ 155us) write_op.cc:270] Start()
0808 13:35:35.874348 (+ 116us) write_op.cc:276] Timestamp: P: 1786196135874112 usec, L: 0
0808 13:35:35.874366 (+ 18us) op_driver.cc:331] REPLICATION: starting
0808 13:35:35.877369 (+ 3003us) log.cc:842] Serialized 433 byte log entry
0808 13:35:35.888709 (+ 11340us) op_driver.cc:444] REPLICATION: finished
0808 13:35:35.889258 (+ 549us) write_op.cc:301] APPLY: starting
0808 13:35:35.889359 (+ 101us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:35.889566 (+ 207us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:35.889786 (+ 220us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:35.890641 (+ 855us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:35.890936 (+ 295us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:35.891033 (+ 97us) write_op.cc:312] APPLY: finished
0808 13:35:35.891347 (+ 314us) log.cc:842] Serialized 29 byte log entry
0808 13:35:35.892133 (+ 786us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.892254 (+ 121us) write_op.cc:454] Released schema lock
0808 13:35:35.892316 (+ 62us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:35.892598 (+ 282us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.892628 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":313,"apply.run_cpu_time_us":3114,"apply.run_wall_time_us":3696,"num_ops":2,"prepare.queue_time_us":2436,"raft.queue_time_us":343,"raft.run_cpu_time_us":825,"raft.run_wall_time_us":825,"replication_time_us":14248}]]}
I20260808 13:35:35.897154 4836 master_service.cc:946] ReplaceTablet: received request to replace tablet e00f2e0b98af431d99894e8778257bc4 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:35.909781 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 308) took 3583 ms. Trace:
I20260808 13:35:35.910107 7199 rpcz_store.cc:276] 0808 13:35:32.326372 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.326496 (+ 124us) service_pool.cc:227] Handling call
0808 13:35:35.854194 (+3527698us) ts_tablet_manager.cc:1145] Deleting tablet 20113491b108476b95b8f32b69176d0e
0808 13:35:35.854250 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.909714 (+ 55464us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3526971}
W20260808 13:35:35.911762 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 20113491b108476b95b8f32b69176d0e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 20113491b108476b95b8f32b69176d0e already in progress: creating tablet
W20260808 13:35:35.911784 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 118093us
I20260808 13:35:35.913393 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_642fc6953dc8471cbd814ae6fda0cc6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:35.914793 7014 ts_tablet_manager.cc:1918] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:35.915192 7014 log.cc:1196] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/fc9b2e69fe27431dac9eb5ecb9e3e23e
W20260808 13:35:35.915253 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_642fc6953dc8471cbd814ae6fda0cc6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:35.917181 7014 ts_tablet_manager.cc:1939] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:35.919426 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 309) took 2685 ms. Trace:
I20260808 13:35:35.919857 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet fc9b2e69fe27431dac9eb5ecb9e3e23e (table default.Table_13f833fb48504c76865676e23461e6e5 [id=0a8dbdef483f48b68019ea98b335a11e]) successfully deleted
I20260808 13:35:35.919742 7199 rpcz_store.cc:276] 0808 13:35:33.234223 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.234410 (+ 187us) service_pool.cc:227] Handling call
0808 13:35:35.911096 (+2676686us) ts_tablet_manager.cc:1145] Deleting tablet 3451d72548a140c68a6ccc2b566a09b1
0808 13:35:35.911144 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.919372 (+ 8228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2676124}
I20260808 13:35:35.920650 7014 tablet_replica.cc:330] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:35.921370 7014 raft_consensus.cc:2229] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 13:35:35.921537 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 3451d72548a140c68a6ccc2b566a09b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3451d72548a140c68a6ccc2b566a09b1 already in progress: creating tablet
I20260808 13:35:35.921797 7014 raft_consensus.cc:2258] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:35.922950 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b23ea067bd8149539074d42c1596a2aa" } modify_external_catalogs: true
I20260808 13:35:35.923635 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b23ea067bd8149539074d42c1596a2aa" } modify_external_catalogs: true
I20260808 13:35:35.927446 5640 raft_consensus.cc:2454] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:35.931324 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1847d0fecb843cdbc833ad674a503a4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:35.943842 4838 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 393) took 1692 ms. Trace:
I20260808 13:35:35.894502 6046 tablet_bootstrap.cc:492] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:35.946758 7014 ts_tablet_manager.cc:1905] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:35.909463 7702 raft_consensus.cc:504] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:35.965250 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 310) took 2727 ms. Trace:
I20260808 13:35:35.964991 7690 consensus_queue.cc:1035] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:35.965665 7199 rpcz_store.cc:276] 0808 13:35:33.238129 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.238263 (+ 134us) service_pool.cc:227] Handling call
0808 13:35:35.920518 (+2682255us) ts_tablet_manager.cc:1145] Deleting tablet 1ba5fa465ec04bbb801a821a71a8f87d
0808 13:35:35.920557 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.965192 (+ 44635us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2681649}
I20260808 13:35:35.970641 4837 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 394) took 1613 ms. Trace:
I20260808 13:35:35.971081 4837 rpcz_store.cc:276] 0808 13:35:34.357121 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.748351 (+1391230us) service_pool.cc:227] Handling call
0808 13:35:35.751313 (+ 2962us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:35.788799 (+ 37486us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:35.969918 (+181119us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:35.970288 (+ 370us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:35.970557 (+ 269us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:35.904694 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:35.905127 (+ 433us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:35.905153 (+ 26us) write_op.cc:435] Acquired schema lock
0808 13:35:35.905169 (+ 16us) tablet.cc:660] Decoding operations
0808 13:35:35.907014 (+ 1845us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:35.907057 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:35.907081 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:35.907281 (+ 200us) tablet.cc:699] Row locks acquired
0808 13:35:35.907295 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:35.907404 (+ 109us) write_op.cc:270] Start()
0808 13:35:35.907484 (+ 80us) write_op.cc:276] Timestamp: P: 1786196135907294 usec, L: 0
0808 13:35:35.907501 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:35.907957 (+ 456us) log.cc:842] Serialized 433 byte log entry
0808 13:35:35.929293 (+ 21336us) op_driver.cc:444] REPLICATION: finished
0808 13:35:35.934287 (+ 4994us) write_op.cc:301] APPLY: starting
0808 13:35:35.934385 (+ 98us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:35.934595 (+ 210us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:35.935872 (+ 1277us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:35.936612 (+ 740us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:35.936834 (+ 222us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:35.936913 (+ 79us) write_op.cc:312] APPLY: finished
0808 13:35:35.961754 (+ 24841us) log.cc:842] Serialized 29 byte log entry
0808 13:35:35.967433 (+ 5679us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.969005 (+ 1572us) write_op.cc:454] Released schema lock
0808 13:35:35.969221 (+ 216us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:35.969472 (+ 251us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.969510 (+ 38us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4687,"apply.run_cpu_time_us":3089,"apply.run_wall_time_us":35477,"mutex_wait_us":133371,"num_ops":2,"prepare.queue_time_us":114511,"prepare.run_cpu_time_us":2114,"prepare.run_wall_time_us":38065,"raft.queue_time_us":91,"raft.run_cpu_time_us":1276,"raft.run_wall_time_us":1276,"replication_time_us":21687}]]}
I20260808 13:35:35.990053 7013 ts_tablet_manager.cc:1918] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:35.993417 7013 log.cc:1196] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/e00f2e0b98af431d99894e8778257bc4
I20260808 13:35:35.994517 6046 tablet_bootstrap.cc:654] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:36.001921 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 1ba5fa465ec04bbb801a821a71a8f87d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ba5fa465ec04bbb801a821a71a8f87d already in progress: creating tablet
I20260808 13:35:35.995398 7013 ts_tablet_manager.cc:1939] T e00f2e0b98af431d99894e8778257bc4 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:36.006031 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 313) took 2697 ms. Trace:
I20260808 13:35:36.006433 7199 rpcz_store.cc:276] 0808 13:35:33.308272 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.308422 (+ 150us) service_pool.cc:227] Handling call
0808 13:35:35.967097 (+2658675us) ts_tablet_manager.cc:1145] Deleting tablet 61dfd47db0a74ed8b7228b559dd68459
0808 13:35:35.967138 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.005964 (+ 38826us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2657988}
W20260808 13:35:36.007773 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 61dfd47db0a74ed8b7228b559dd68459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 61dfd47db0a74ed8b7228b559dd68459 already in progress: creating tablet
I20260808 13:35:36.016793 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 326) took 2581 ms. Trace:
I20260808 13:35:36.017124 7199 rpcz_store.cc:276] 0808 13:35:33.435582 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.435804 (+ 222us) service_pool.cc:227] Handling call
0808 13:35:36.008790 (+2572986us) ts_tablet_manager.cc:1145] Deleting tablet cb7f6487e28f4aaa94d448be71f8a3c2
0808 13:35:36.008829 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.016734 (+ 7905us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2572329}
W20260808 13:35:36.019969 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet cb7f6487e28f4aaa94d448be71f8a3c2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb7f6487e28f4aaa94d448be71f8a3c2 already in progress: creating tablet
I20260808 13:35:36.021566 5283 leader_election.cc:302] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:36.023137 7690 raft_consensus.cc:482] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:36.023916 7690 raft_consensus.cc:504] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.014926 7702 leader_election.cc:288] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:36.026909 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 329) took 2219 ms. Trace:
I20260808 13:35:36.027104 7716 raft_consensus.cc:3037] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:36.027307 7199 rpcz_store.cc:276] 0808 13:35:33.807946 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.810729 (+ 2783us) service_pool.cc:227] Handling call
0808 13:35:36.017922 (+2207193us) ts_tablet_manager.cc:1145] Deleting tablet 293e9540498e420d8ec873472d0d29fc
0808 13:35:36.017968 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.026847 (+ 8879us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2206644}
I20260808 13:35:36.028137 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9268f5f6dee4112932e106010a3c2be" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:36.029892 5551 leader_election.cc:341] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:36.034219 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 293e9540498e420d8ec873472d0d29fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 293e9540498e420d8ec873472d0d29fc already in progress: creating tablet
I20260808 13:35:36.034806 7014 ts_tablet_manager.cc:1918] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:36.035229 7014 log.cc:1196] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/1f9bc338093f403881d329a7e913c9f7
I20260808 13:35:36.036134 7014 ts_tablet_manager.cc:1939] T 1f9bc338093f403881d329a7e913c9f7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:36.037827 7690 leader_election.cc:288] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:36.055109 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 1f9bc338093f403881d329a7e913c9f7 (table default.Table_49299116c37c4b9dbb509dbc8efae85e [id=d88dc1106b9b4e1eb18dc680d15fdfcf]) successfully deleted
I20260808 13:35:36.056881 5283 leader_election.cc:302] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:36.057565 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 2cd6945ecccc4684a3fa5b2a1f07057f because the tablet was not found. No further retry: Not found: Tablet not found: 2cd6945ecccc4684a3fa5b2a1f07057f
I20260808 13:35:36.058301 7727 raft_consensus.cc:2726] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260808 13:35:36.059949 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 127e8d0323954ef788018bcb91582951 because the tablet was not found. No further retry: Not found: Tablet not found: 127e8d0323954ef788018bcb91582951
W20260808 13:35:36.062788 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet a8e18e51c2e24d3caa3d29e6645e46ae because the tablet was not found. No further retry: Not found: Tablet not found: a8e18e51c2e24d3caa3d29e6645e46ae
W20260808 13:35:36.063673 5772 consensus_peers.cc:487] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 69642fdba1f145438b04eae9f8154101. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:36.064333 7014 tablet_replica.cc:330] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:36.065337 7014 raft_consensus.cc:2229] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:36.067523 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9268f5f6dee4112932e106010a3c2be" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:36.068423 5883 raft_consensus.cc:2454] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
W20260808 13:35:36.071882 5551 consensus_peers.cc:487] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:36.072576 7014 raft_consensus.cc:2258] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:36.072667 5550 leader_election.cc:302] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:36.073851 7562 raft_consensus.cc:2781] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:36.074386 7562 raft_consensus.cc:482] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:36.075619 7562 raft_consensus.cc:3037] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:36.076907 7014 ts_tablet_manager.cc:1905] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:36.078424 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 335) took 1181 ms. Trace:
I20260808 13:35:36.078805 7199 rpcz_store.cc:276] 0808 13:35:34.896539 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.896685 (+ 146us) service_pool.cc:227] Handling call
0808 13:35:36.028232 (+1131547us) ts_tablet_manager.cc:1145] Deleting tablet 6e70a5cf88d94b36913a6d1ccfcb979a
0808 13:35:36.067184 (+ 38952us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.078295 (+ 11111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1130591}
I20260808 13:35:36.080330 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 334) took 1184 ms. Trace:
I20260808 13:35:36.080670 7199 rpcz_store.cc:276] 0808 13:35:34.895887 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.898646 (+ 2759us) service_pool.cc:227] Handling call
0808 13:35:36.079735 (+1181089us) ts_tablet_manager.cc:1145] Deleting tablet 1f0a6d9bd2c74d0790badd7977f027e3
0808 13:35:36.079775 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.080266 (+ 491us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1180484}
I20260808 13:35:36.086071 7434 raft_consensus.cc:482] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:36.086874 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 6e70a5cf88d94b36913a6d1ccfcb979a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e70a5cf88d94b36913a6d1ccfcb979a already in progress: creating tablet
W20260808 13:35:36.087687 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 1f0a6d9bd2c74d0790badd7977f027e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f0a6d9bd2c74d0790badd7977f027e3 already in progress: creating tablet
I20260808 13:35:36.089691 6026 tablet_bootstrap.cc:492] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:36.090117 6026 ts_tablet_manager.cc:1397] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.622s user 0.014s sys 0.003s
I20260808 13:35:36.089457 7434 raft_consensus.cc:504] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.096350 6033 tablet_bootstrap.cc:492] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:36.096801 6033 ts_tablet_manager.cc:1397] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.517s user 0.013s sys 0.008s
I20260808 13:35:36.097294 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16a6552c871b4b81b6520cb084d359a2" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:36.097908 5383 raft_consensus.cc:2454] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:36.102162 5551 leader_election.cc:302] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:36.102468 6033 raft_consensus.cc:348] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.103165 6033 raft_consensus.cc:374] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:36.103430 6033 raft_consensus.cc:729] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:36.103588 7434 leader_election.cc:288] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:36.104331 7434 raft_consensus.cc:2781] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:36.103986 6033 consensus_queue.cc:260] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.104856 7434 raft_consensus.cc:482] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:36.105316 7434 raft_consensus.cc:3037] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:36.111426 4836 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a384cb98038948a2aa2ea653c22ac2eb" } new_table_name: "default.Table_e024f29e457447139994e4d9c43a41ce" modify_external_catalogs: true
I20260808 13:35:36.115206 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_7bf999991a5c4f998994907960315c0e" } new_table_name: "default.Table_2d75fe6a74ad43789e515c67b5167ab3" modify_external_catalogs: true
I20260808 13:35:36.016955 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 312) took 3983 ms. Trace:
I20260808 13:35:36.117395 7013 rpcz_store.cc:276] 0808 13:35:32.034057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.034508 (+ 451us) service_pool.cc:227] Handling call
0808 13:35:35.723688 (+3689180us) ts_tablet_manager.cc:1145] Deleting tablet e00f2e0b98af431d99894e8778257bc4
0808 13:35:35.723788 (+ 100us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:35.883418 (+159630us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:35.989843 (+106425us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:36.016894 (+ 27051us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3684758}
I20260808 13:35:36.120609 7562 raft_consensus.cc:504] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:36.129487 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 65177us
I20260808 13:35:36.130187 5772 leader_election.cc:302] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:35.944377 4838 rpcz_store.cc:276] 0808 13:35:34.251476 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.648727 (+1397251us) service_pool.cc:227] Handling call
0808 13:35:35.750701 (+101974us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:35.752857 (+ 2156us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:35.942888 (+190031us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.942918 (+ 30us) write_op.cc:454] Released schema lock
0808 13:35:35.943130 (+ 212us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:35.943477 (+ 347us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:35.943779 (+ 302us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:35.793409 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:35.793851 (+ 442us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:35.793888 (+ 37us) write_op.cc:435] Acquired schema lock
0808 13:35:35.793900 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:35.794335 (+ 435us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:35.794395 (+ 60us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:35.794427 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:35.794721 (+ 294us) tablet.cc:699] Row locks acquired
0808 13:35:35.794741 (+ 20us) write_op.cc:260] PREPARE: finished
0808 13:35:35.794879 (+ 138us) write_op.cc:270] Start()
0808 13:35:35.794987 (+ 108us) write_op.cc:276] Timestamp: P: 1786196135794768 usec, L: 0
0808 13:35:35.795011 (+ 24us) op_driver.cc:331] REPLICATION: starting
0808 13:35:35.795904 (+ 893us) log.cc:842] Serialized 433 byte log entry
0808 13:35:35.811479 (+ 15575us) op_driver.cc:444] REPLICATION: finished
0808 13:35:35.811933 (+ 454us) write_op.cc:301] APPLY: starting
0808 13:35:35.812050 (+ 117us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:35.812282 (+ 232us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:35.812558 (+ 276us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:35.813446 (+ 888us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:35.813724 (+ 278us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:35.813826 (+ 102us) write_op.cc:312] APPLY: finished
0808 13:35:35.814159 (+ 333us) log.cc:842] Serialized 29 byte log entry
0808 13:35:35.814471 (+ 312us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:35.815311 (+ 840us) write_op.cc:454] Released schema lock
0808 13:35:35.815416 (+ 105us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":252,"apply.run_cpu_time_us":3157,"apply.run_wall_time_us":3767,"num_ops":2,"prepare.queue_time_us":39267,"prepare.run_cpu_time_us":2837,"prepare.run_wall_time_us":2837,"raft.queue_time_us":1676,"raft.run_cpu_time_us":705,"raft.run_wall_time_us":705,"replication_time_us":16387}]]}
I20260808 13:35:36.139595 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:36.141866 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet e00f2e0b98af431d99894e8778257bc4 (table default.Table_8523d4a015194c64b22f66a22b0dcc06 [id=b285a956530e4fb4a5d1632a45d87c0e]) successfully deleted
I20260808 13:35:36.144435 7716 raft_consensus.cc:2726] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:36.148769 4835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet e5fb1d373c4d486ca3815023897721a2
I20260808 13:35:36.150035 4835 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 398) took 1118 ms. Trace:
I20260808 13:35:36.150312 4835 rpcz_store.cc:276] 0808 13:35:35.031342 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.887298 (+855956us) service_pool.cc:227] Handling call
0808 13:35:35.892626 (+ 5328us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:35.893215 (+ 589us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:35.893328 (+ 113us) catalog_manager.cc:2761] Locking tablets
0808 13:35:35.895861 (+ 2533us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:36.134824 (+238963us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:36.148593 (+ 13769us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:36.148696 (+ 103us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:36.149985 (+ 1289us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:36.083237 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:36.083676 (+ 439us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:36.083699 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:36.083783 (+ 84us) tablet.cc:660] Decoding operations
0808 13:35:36.084167 (+ 384us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:36.084228 (+ 61us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:36.084265 (+ 37us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:36.084608 (+ 343us) tablet.cc:699] Row locks acquired
0808 13:35:36.084623 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:36.084760 (+ 137us) write_op.cc:270] Start()
0808 13:35:36.084869 (+ 109us) write_op.cc:276] Timestamp: P: 1786196136084690 usec, L: 0
0808 13:35:36.084886 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:36.085574 (+ 688us) log.cc:842] Serialized 725 byte log entry
0808 13:35:36.126399 (+ 40825us) op_driver.cc:444] REPLICATION: finished
0808 13:35:36.130156 (+ 3757us) write_op.cc:301] APPLY: starting
0808 13:35:36.130248 (+ 92us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:36.130389 (+ 141us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:36.130525 (+ 136us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:36.130962 (+ 437us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:36.131135 (+ 173us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:36.131205 (+ 70us) write_op.cc:312] APPLY: finished
0808 13:35:36.131484 (+ 279us) log.cc:842] Serialized 29 byte log entry
0808 13:35:36.134030 (+ 2546us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:36.134127 (+ 97us) write_op.cc:454] Released schema lock
0808 13:35:36.134174 (+ 47us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:36.135291 (+ 1117us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:36.135421 (+ 130us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3567,"apply.run_cpu_time_us":2185,"apply.run_wall_time_us":6283,"num_ops":2,"prepare.queue_time_us":143591,"prepare.run_cpu_time_us":2753,"prepare.run_wall_time_us":61467,"raft.queue_time_us":169,"raft.run_cpu_time_us":888,"raft.run_wall_time_us":889,"replication_time_us":41440}]]}
I20260808 13:35:36.154783 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9268f5f6dee4112932e106010a3c2be" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:36.154407 7434 raft_consensus.cc:504] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:36.155982 5551 leader_election.cc:341] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:36.157044 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fb1d373c4d486ca3815023897721a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:35 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:36.158274 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fb1d373c4d486ca3815023897721a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:35 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:36.159785 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 337) took 1192 ms. Trace:
I20260808 13:35:36.160025 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16a6552c871b4b81b6520cb084d359a2" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:36.160189 7199 rpcz_store.cc:276] 0808 13:35:34.967540 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.968192 (+ 652us) service_pool.cc:227] Handling call
0808 13:35:36.081564 (+1113372us) ts_tablet_manager.cc:1145] Deleting tablet a8e18e51c2e24d3caa3d29e6645e46ae
0808 13:35:36.159340 (+ 77776us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.159664 (+ 324us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1112235}
I20260808 13:35:36.160723 5883 raft_consensus.cc:2454] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:36.161161 7199 tablet_replica.cc:330] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:36.161924 7199 raft_consensus.cc:2229] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:36.163295 5826 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 343) took 1024 ms. Trace:
W20260808 13:35:36.163823 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet a8e18e51c2e24d3caa3d29e6645e46ae because the tablet was not found. No further retry: Not found: Tablet not found: a8e18e51c2e24d3caa3d29e6645e46ae
I20260808 13:35:36.163771 5826 rpcz_store.cc:276] 0808 13:35:35.139269 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.171762 (+ 32493us) service_pool.cc:227] Handling call
0808 13:35:35.554667 (+382905us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:35.554770 (+ 103us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:35.954853 (+400083us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:36.066708 (+111855us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:36.163223 (+ 96515us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:36.170346 6046 tablet_bootstrap.cc:492] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:36.170931 6046 ts_tablet_manager.cc:1397] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.277s user 0.012s sys 0.005s
I20260808 13:35:36.173487 5598 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fb1d373c4d486ca3815023897721a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:35 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:36.187188 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9268f5f6dee4112932e106010a3c2be" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:36.187995 5883 raft_consensus.cc:3037] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:36.190858 7716 raft_consensus.cc:482] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:36.191475 7716 raft_consensus.cc:504] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:36.192487 4836 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4747a38c520d40848f4d3c261499aa88 consensus state has no leader
I20260808 13:35:36.194794 7716 leader_election.cc:288] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:36.195364 4823 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet d9268f5f6dee4112932e106010a3c2be consensus state has no leader
I20260808 13:35:36.196612 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264ee45f8dfa4a05b0e5e812ce1b622c" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:36.198530 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16a6552c871b4b81b6520cb084d359a2" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:36.199090 5383 raft_consensus.cc:3037] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:36.200722 7434 leader_election.cc:288] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:36.203387 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_e4788552c1a8457a8d34951f6827c5e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:36.205224 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4788552c1a8457a8d34951f6827c5e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:36.209550 5383 raft_consensus.cc:2454] T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:36.210566 5551 leader_election.cc:302] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:36.211866 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16a6552c871b4b81b6520cb084d359a2" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:36.212709 5882 raft_consensus.cc:3037] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:36.214412 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 9423b380a62c467895e7fdff14756586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:36.214908 7434 raft_consensus.cc:2781] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:36.147074 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a446b3e7bb7944b0a5b119dd6a1b6d7c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:36.216403 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a446b3e7bb7944b0a5b119dd6a1b6d7c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:36.218712 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f2e0b98af431d99894e8778257bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:36.219162 7434 raft_consensus.cc:686] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:36.094261 6026 raft_consensus.cc:348] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.219851 6026 raft_consensus.cc:374] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:36.220165 6026 raft_consensus.cc:729] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:36.220973 6026 consensus_queue.cc:260] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.223533 5883 raft_consensus.cc:2454] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:36.224845 5550 leader_election.cc:302] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:36.225558 7690 raft_consensus.cc:2781] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:36.226094 7690 raft_consensus.cc:686] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:36.227061 7690 consensus_queue.cc:237] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.229835 7562 leader_election.cc:288] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:36.240191 6033 ts_tablet_manager.cc:1428] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.143s user 0.007s sys 0.000s
I20260808 13:35:36.241240 6033 tablet_bootstrap.cc:492] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:36.243736 7014 ts_tablet_manager.cc:1918] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260808 13:35:36.243840 5771 leader_election.cc:341] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:36.244132 7014 log.cc:1196] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/9423b380a62c467895e7fdff14756586
W20260808 13:35:36.244647 4833 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 53) took 1820 ms (client timeout 1000 ms). Trace:
I20260808 13:35:36.221668 7434 consensus_queue.cc:237] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:36.245074 4833 rpcz_store.cc:269] 0808 13:35:34.424230 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.846228 (+421998us) service_pool.cc:227] Handling call
0808 13:35:34.911360 (+ 65132us) spinlock_profiling.cc:243] Waited 47.3 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:36.244571 (+1333211us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":67614,"reported_tablets":29,"spinlock_wait_cycles":108750720}
I20260808 13:35:36.247854 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6b9bc590a69746909348fbcbb7856cdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:36.249040 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d8d5a3ca724141b58e4e2cea23be68" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:36.244848 7014 ts_tablet_manager.cc:1939] T 9423b380a62c467895e7fdff14756586 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:36.251372 6046 raft_consensus.cc:348] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.262631 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 313) took 4224 ms. Trace:
I20260808 13:35:36.262972 7013 rpcz_store.cc:276] 0808 13:35:32.038082 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.038214 (+ 132us) service_pool.cc:227] Handling call
0808 13:35:36.141679 (+4103465us) ts_tablet_manager.cc:1145] Deleting tablet 5d388589d386408bb10ca030b87af787
0808 13:35:36.262293 (+120614us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.262555 (+ 262us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4102706}
I20260808 13:35:36.263940 7013 tablet_replica.cc:330] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:36.264628 7013 raft_consensus.cc:2229] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:36.265118 7013 raft_consensus.cc:2258] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:36.267453 7013 ts_tablet_manager.cc:1905] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:36.253053 6046 raft_consensus.cc:374] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:36.269172 6046 raft_consensus.cc:729] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:36.269928 6046 consensus_queue.cc:260] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.273530 5882 raft_consensus.cc:2454] T 16a6552c871b4b81b6520cb084d359a2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
W20260808 13:35:36.273957 5771 leader_election.cc:341] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:36.277173 7199 raft_consensus.cc:2258] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:36.279366 7199 ts_tablet_manager.cc:1905] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:36.280014 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 149903us
W20260808 13:35:36.281253 5772 consensus_peers.cc:487] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc9b2e69fe27431dac9eb5ecb9e3e23e. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:36.283583 7387 raft_consensus.cc:2781] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:36.284102 7387 raft_consensus.cc:686] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:36.285053 7387 consensus_queue.cc:237] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.288152 6046 ts_tablet_manager.cc:1428] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.117s user 0.005s sys 0.001s
I20260808 13:35:36.288988 6046 tablet_bootstrap.cc:492] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:36.223470 6026 ts_tablet_manager.cc:1428] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.133s user 0.004s sys 0.004s
I20260808 13:35:36.295754 6026 tablet_bootstrap.cc:492] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:36.302392 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d8d5a3ca724141b58e4e2cea23be68" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:36.303614 5629 raft_consensus.cc:2454] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:36.306211 7560 consensus_queue.cc:1035] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:36.254529 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 9423b380a62c467895e7fdff14756586 (table default.Table_1fe1ec319d834d319cd9c35e801b1c91 [id=0f9219b97bfc4351ad5b584cfdec058f]) successfully deleted
W20260808 13:35:36.314152 4821 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 59641.2us
I20260808 13:35:36.314380 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264ee45f8dfa4a05b0e5e812ce1b622c" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:36.314850 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 31fc9fdb5ef543da833f9af6ca5807a9 because the tablet was not found. No further retry: Not found: Tablet not found: 31fc9fdb5ef543da833f9af6ca5807a9
I20260808 13:35:36.315165 5882 raft_consensus.cc:2379] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
W20260808 13:35:36.315434 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 69642fdba1f145438b04eae9f8154101 because the tablet was not found. No further retry: Not found: Tablet not found: 69642fdba1f145438b04eae9f8154101
W20260808 13:35:36.316037 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet e5fb1d373c4d486ca3815023897721a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5fb1d373c4d486ca3815023897721a2 already in progress: creating tablet
W20260808 13:35:36.271324 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 5d388589d386408bb10ca030b87af787 because the tablet was not found. No further retry: Not found: Tablet not found: 5d388589d386408bb10ca030b87af787
W20260808 13:35:36.316648 4820 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 45364.9us
I20260808 13:35:36.326092 5283 leader_election.cc:302] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:36.327442 7727 raft_consensus.cc:3037] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:36.362931 7013 ts_tablet_manager.cc:1918] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:36.362936 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_0913077ace074f11bf8a6222338489c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:36.364830 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0913077ace074f11bf8a6222338489c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:36.365754 7758 tablet.cc:1722] T c4c083c8e19d4279af600670f5d1d78d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260808 13:35:36.372560 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b9bc590a69746909348fbcbb7856cdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:36.375128 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 46323.6us
I20260808 13:35:36.375875 5772 leader_election.cc:302] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:36.376850 7013 log.cc:1196] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/e992a0294c404cfb9541b57782a05fc9
I20260808 13:35:36.381672 7013 ts_tablet_manager.cc:1939] T e992a0294c404cfb9541b57782a05fc9 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:36.385181 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 316) took 4331 ms. Trace:
I20260808 13:35:36.364573 7727 raft_consensus.cc:2726] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:36.386260 6046 tablet_bootstrap.cc:654] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:36.404131 4838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6eb3cce7943e47dc9749d45cfae01235" } modify_external_catalogs: true
I20260808 13:35:36.404955 4838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6eb3cce7943e47dc9749d45cfae01235" } modify_external_catalogs: true
I20260808 13:35:36.388165 7013 rpcz_store.cc:276] 0808 13:35:32.053908 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.054009 (+ 101us) service_pool.cc:227] Handling call
0808 13:35:36.263759 (+4209750us) ts_tablet_manager.cc:1145] Deleting tablet e992a0294c404cfb9541b57782a05fc9
0808 13:35:36.263837 (+ 78us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.275689 (+ 11852us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:36.362627 (+ 86938us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:36.385114 (+ 22487us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4209165}
I20260808 13:35:36.414749 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 315) took 4361 ms. Trace:
I20260808 13:35:36.414932 7704 raft_consensus.cc:2781] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:36.415632 7704 raft_consensus.cc:482] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:36.415098 7719 consensus_queue.cc:1035] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:36.416122 7704 raft_consensus.cc:3037] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:36.418107 6026 tablet_bootstrap.cc:654] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:36.415465 7013 rpcz_store.cc:276] 0808 13:35:32.053285 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.053604 (+ 319us) service_pool.cc:227] Handling call
0808 13:35:36.413857 (+4360253us) ts_tablet_manager.cc:1145] Deleting tablet a541ec35b1a44109866fabc953f14cc6
0808 13:35:36.414018 (+ 161us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.414644 (+ 626us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4357862}
I20260808 13:35:36.423965 7013 tablet_replica.cc:330] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
W20260808 13:35:36.427882 5772 consensus_peers.cc:487] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f9bc338093f403881d329a7e913c9f7. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:36.411645 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet e992a0294c404cfb9541b57782a05fc9 (table default.Table_6f4c6712b74748f58390a05ec941a26a [id=2b4fc1c958044df7939df70bc136d956]) successfully deleted
W20260808 13:35:36.438524 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet a541ec35b1a44109866fabc953f14cc6 because the tablet was not found. No further retry: Not found: Tablet not found: a541ec35b1a44109866fabc953f14cc6
I20260808 13:35:36.313652 5832 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 348) took 1130 ms. Trace:
I20260808 13:35:36.444509 5832 rpcz_store.cc:276] 0808 13:35:35.183621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.183756 (+ 135us) service_pool.cc:227] Handling call
0808 13:35:35.505444 (+321688us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b5800001898. stack: 00007f19df247033 00007f19df246f30 00007f19dedc8534 00000000004db2b4 00000000004db249 00007f19df0bdf11 00007f19df241f98 00007f19df241ebb 00007f19e81018b9 00007f19e71355d8 00007f19e71f52a0 00007f19e71a6869 00007f19e5898c44 00007f19e5898bd2 00007f19e5898b01 00007f19e5898a7c
0808 13:35:35.505534 (+ 90us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:35.505742 (+ 208us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:36.052622 (+546880us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:36.312702 (+260080us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:36.313521 (+ 819us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":91378048}
I20260808 13:35:36.474332 7013 raft_consensus.cc:2229] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:36.475557 7013 raft_consensus.cc:2258] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:36.478223 7013 ts_tablet_manager.cc:1905] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:36.487066 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_7cfe2eb0d78341a0816f9b5e14316b4f" } new_table_name: "default.Table_cd53ce0f4bbd47f5a510f440e7e066c8" modify_external_catalogs: true
W20260808 13:35:36.511560 4823 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c -> Peer 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135): Couldn't send request to peer 87e8b888041b4135af55792050defcb5. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260808 13:35:36.545795 6026 tablet_bootstrap.cc:492] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:36.546356 6026 ts_tablet_manager.cc:1397] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.251s user 0.011s sys 0.013s
I20260808 13:35:36.550766 6026 raft_consensus.cc:348] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.557865 6026 raft_consensus.cc:374] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:36.558305 6026 raft_consensus.cc:729] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:36.559269 6026 consensus_queue.cc:260] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.562911 6026 ts_tablet_manager.cc:1428] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
W20260808 13:35:36.564380 4833 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 24096c0c30da4f2ea643fcdd5791a927 consensus state has no leader
I20260808 13:35:36.566536 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_034b94572e274eff86802d9cae6c907d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:36.568346 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_034b94572e274eff86802d9cae6c907d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:36.568832 6026 tablet_bootstrap.cc:492] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:36.605523 6026 tablet_bootstrap.cc:654] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:36.608458 6033 tablet_bootstrap.cc:654] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:36.629560 6026 tablet_bootstrap.cc:492] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:36.630012 6026 ts_tablet_manager.cc:1397] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.061s user 0.013s sys 0.006s
I20260808 13:35:36.634545 6026 raft_consensus.cc:348] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.636204 6026 raft_consensus.cc:374] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:36.636441 6026 raft_consensus.cc:729] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:36.637167 6026 consensus_queue.cc:260] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.640000 6026 ts_tablet_manager.cc:1428] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260808 13:35:36.638170 7704 raft_consensus.cc:504] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.669934 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d8d5a3ca724141b58e4e2cea23be68" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:36.683415 5771 leader_election.cc:341] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:36.719277 6026 tablet_bootstrap.cc:492] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:36.728935 6046 tablet_bootstrap.cc:492] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:36.729569 6046 ts_tablet_manager.cc:1397] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.441s user 0.011s sys 0.014s
I20260808 13:35:36.736442 6026 tablet_bootstrap.cc:654] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:36.745607 4832 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]). Expected version 1 got 0
I20260808 13:35:36.747835 4832 catalog_manager.cc:5495] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } }
I20260808 13:35:36.749042 4832 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:36.751258 4832 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:36.755249 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 73e2c53a99ac4ce0a0e0c0c4d5ef25b3 (DEFAULT_TABLE table=default.Table_51d39ab358f3470084d7cd6e270eee0c [id=bbe62ee3dd5d494d935378ada5650363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.757313 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e2c53a99ac4ce0a0e0c0c4d5ef25b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.765961 7704 leader_election.cc:288] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:36.759449 6026 tablet_bootstrap.cc:492] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:36.766930 6026 ts_tablet_manager.cc:1397] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.048s user 0.013s sys 0.006s
I20260808 13:35:36.768013 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d8d5a3ca724141b58e4e2cea23be68" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:36.768616 5630 raft_consensus.cc:3037] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:36.778092 5630 raft_consensus.cc:2454] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:36.769814 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 56d8d5a3ca724141b58e4e2cea23be68
I20260808 13:35:36.780561 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d8d5a3ca724141b58e4e2cea23be68 with delete_type TABLET_DATA_DELETED (replaced by tablet 14a9373000e24e75858e6d7457587264) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:36.782078 7199 ts_tablet_manager.cc:1918] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:36.782421 7199 log.cc:1196] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/2cd6945ecccc4684a3fa5b2a1f07057f
I20260808 13:35:36.734081 6046 raft_consensus.cc:348] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.783018 6046 raft_consensus.cc:374] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:36.783298 6046 raft_consensus.cc:729] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:36.783974 6046 consensus_queue.cc:260] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.785219 6026 raft_consensus.cc:348] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.785925 6026 raft_consensus.cc:374] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:36.785969 5772 leader_election.cc:302] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:36.786193 6026 raft_consensus.cc:729] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:36.786863 6026 consensus_queue.cc:260] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.789528 6026 ts_tablet_manager.cc:1428] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.022s user 0.004s sys 0.003s
I20260808 13:35:36.797195 6026 tablet_bootstrap.cc:492] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:36.796815 7690 consensus_queue.cc:1035] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:36.799343 4832 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:36.800769 7704 raft_consensus.cc:2781] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:36.801561 7704 raft_consensus.cc:686] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:36.803609 7704 consensus_queue.cc:237] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.810099 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet a11a2666f0b44d70a08a9195e8fcf69d
I20260808 13:35:36.811755 5832 tablet_service.cc:1514] Processing DeleteTablet for tablet a11a2666f0b44d70a08a9195e8fcf69d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:36 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:36.812444 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet a11a2666f0b44d70a08a9195e8fcf69d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:36 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:36.818006 5598 tablet_service.cc:1514] Processing DeleteTablet for tablet a11a2666f0b44d70a08a9195e8fcf69d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:36 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:36.819161 5592 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d8d5a3ca724141b58e4e2cea23be68 with delete_type TABLET_DATA_DELETED (replaced by tablet 14a9373000e24e75858e6d7457587264) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:36.819487 4839 catalog_manager.cc:6565] ReplaceTablet: tablet 56d8d5a3ca724141b58e4e2cea23be68 deleted and replaced by tablet 14a9373000e24e75858e6d7457587264
I20260808 13:35:36.819808 4839 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60270 (request call id 397) took 2259 ms. Trace:
I20260808 13:35:36.820469 5832 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d8d5a3ca724141b58e4e2cea23be68 with delete_type TABLET_DATA_DELETED (replaced by tablet 14a9373000e24e75858e6d7457587264) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:36.820104 4839 rpcz_store.cc:276] 0808 13:35:34.560306 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.847153 (+1286847us) service_pool.cc:227] Handling call
0808 13:35:36.819755 (+972602us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:36.675005 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:36.675422 (+ 417us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:36.675444 (+ 22us) write_op.cc:435] Acquired schema lock
0808 13:35:36.675467 (+ 23us) tablet.cc:660] Decoding operations
0808 13:35:36.676472 (+ 1005us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:36.676512 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:36.676534 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:36.676739 (+ 205us) tablet.cc:699] Row locks acquired
0808 13:35:36.676751 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:36.676874 (+ 123us) write_op.cc:270] Start()
0808 13:35:36.677024 (+ 150us) write_op.cc:276] Timestamp: P: 1786196136676813 usec, L: 0
0808 13:35:36.677045 (+ 21us) op_driver.cc:331] REPLICATION: starting
0808 13:35:36.677552 (+ 507us) log.cc:842] Serialized 622 byte log entry
0808 13:35:36.692040 (+ 14488us) op_driver.cc:444] REPLICATION: finished
0808 13:35:36.702340 (+ 10300us) write_op.cc:301] APPLY: starting
0808 13:35:36.702425 (+ 85us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:36.702651 (+ 226us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:36.702865 (+ 214us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:36.703464 (+ 599us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:36.703777 (+ 313us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:36.703879 (+ 102us) write_op.cc:312] APPLY: finished
0808 13:35:36.704222 (+ 343us) log.cc:842] Serialized 29 byte log entry
0808 13:35:36.768338 (+ 64116us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:36.768467 (+ 129us) write_op.cc:454] Released schema lock
0808 13:35:36.768530 (+ 63us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:36.770219 (+ 1689us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:36.770245 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":10042,"apply.run_cpu_time_us":3037,"apply.run_wall_time_us":68271,"num_ops":2,"prepare.queue_time_us":107,"prepare.run_cpu_time_us":2184,"prepare.run_wall_time_us":22399,"raft.queue_time_us":169,"raft.run_cpu_time_us":1084,"raft.run_wall_time_us":1085,"replication_time_us":14919}]]}
I20260808 13:35:36.828649 5592 tablet_service.cc:1467] Processing CreateTablet for tablet 73e2c53a99ac4ce0a0e0c0c4d5ef25b3 (DEFAULT_TABLE table=default.Table_51d39ab358f3470084d7cd6e270eee0c [id=bbe62ee3dd5d494d935378ada5650363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.831003 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e2c53a99ac4ce0a0e0c0c4d5ef25b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.830220 5832 tablet_service.cc:1467] Processing CreateTablet for tablet 73e2c53a99ac4ce0a0e0c0c4d5ef25b3 (DEFAULT_TABLE table=default.Table_51d39ab358f3470084d7cd6e270eee0c [id=bbe62ee3dd5d494d935378ada5650363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.832357 5832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e2c53a99ac4ce0a0e0c0c4d5ef25b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.840513 6026 tablet_bootstrap.cc:654] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:36.841593 6046 ts_tablet_manager.cc:1428] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.112s user 0.005s sys 0.001s
W20260808 13:35:36.847533 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet a11a2666f0b44d70a08a9195e8fcf69d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a11a2666f0b44d70a08a9195e8fcf69d already in progress: creating tablet
I20260808 13:35:36.853626 5337 tablet_service.cc:1467] Processing CreateTablet for tablet fc16da5c85ff4540ad74d332ca962979 (DEFAULT_TABLE table=default.Table_2269d7b97e6a488b9a86e8240b3b197e [id=7a78c3e7fd434fe48e5d0d8faa195f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.855000 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc16da5c85ff4540ad74d332ca962979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.856317 7793 tablet_replica.cc:330] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:36.857991 7793 raft_consensus.cc:2229] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:36.860359 7793 raft_consensus.cc:2258] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:36.864507 6033 tablet_bootstrap.cc:492] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:36.864991 6033 ts_tablet_manager.cc:1397] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.624s user 0.008s sys 0.008s
I20260808 13:35:36.868204 7793 ts_tablet_manager.cc:1905] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:36.876464 5598 tablet_service.cc:1467] Processing CreateTablet for tablet fc16da5c85ff4540ad74d332ca962979 (DEFAULT_TABLE table=default.Table_2269d7b97e6a488b9a86e8240b3b197e [id=7a78c3e7fd434fe48e5d0d8faa195f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.877753 5598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc16da5c85ff4540ad74d332ca962979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.878688 5826 tablet_service.cc:1467] Processing CreateTablet for tablet fc16da5c85ff4540ad74d332ca962979 (DEFAULT_TABLE table=default.Table_2269d7b97e6a488b9a86e8240b3b197e [id=7a78c3e7fd434fe48e5d0d8faa195f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.880744 5826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc16da5c85ff4540ad74d332ca962979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.889494 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb821666b9c4a93af438ccba51c0c7c (DEFAULT_TABLE table=default.Table_7e0bc5965bd54d788347d0d41c53208f [id=296621c6dba14fbc8305eca14e4396a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.891137 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb821666b9c4a93af438ccba51c0c7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.908671 6026 tablet_bootstrap.cc:492] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:36.909176 6026 ts_tablet_manager.cc:1397] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.116s user 0.004s sys 0.013s
I20260808 13:35:36.915185 6033 raft_consensus.cc:348] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.917212 6046 tablet_bootstrap.cc:492] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:36.928529 5351 tablet_service.cc:1467] Processing CreateTablet for tablet fdacfbbaef114ea7a9da62f8a65916f6 (DEFAULT_TABLE table=default.Table_b84003117fd04b17930f7a3ba4e2d05f [id=7f4fa75c41e542928b94ef51cfd2f33a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.929857 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdacfbbaef114ea7a9da62f8a65916f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.921586 5592 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb821666b9c4a93af438ccba51c0c7c (DEFAULT_TABLE table=default.Table_7e0bc5965bd54d788347d0d41c53208f [id=296621c6dba14fbc8305eca14e4396a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.913198 6026 raft_consensus.cc:348] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.930675 6026 raft_consensus.cc:374] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:36.929605 5595 tablet_service.cc:1467] Processing CreateTablet for tablet fdacfbbaef114ea7a9da62f8a65916f6 (DEFAULT_TABLE table=default.Table_b84003117fd04b17930f7a3ba4e2d05f [id=7f4fa75c41e542928b94ef51cfd2f33a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.931107 6026 raft_consensus.cc:729] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:36.931711 6026 consensus_queue.cc:260] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:36.936757 5357 tablet_service.cc:1467] Processing CreateTablet for tablet d5c010e1c8804b7e8e4a58f091cad1d1 (DEFAULT_TABLE table=default.Table_6fc58ab15c1b44c69500f3ad87062bcd [id=562d628bce6b4d96b2013e847c87a8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.938237 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5c010e1c8804b7e8e4a58f091cad1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.938722 6026 ts_tablet_manager.cc:1428] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.029s user 0.003s sys 0.005s
I20260808 13:35:36.972465 5613 tablet_service.cc:1467] Processing CreateTablet for tablet d5c010e1c8804b7e8e4a58f091cad1d1 (DEFAULT_TABLE table=default.Table_6fc58ab15c1b44c69500f3ad87062bcd [id=562d628bce6b4d96b2013e847c87a8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.976601 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 3999ebf74e2e4c3abae914fb3fbe3b15 (DEFAULT_TABLE table=default.Table_b8623fa3d1eb479a96e59b16c329e590 [id=c0f00fb75b204087bd476376ddfdc046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.977643 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3999ebf74e2e4c3abae914fb3fbe3b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.977937 7793 ts_tablet_manager.cc:1918] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:36.978075 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 3999ebf74e2e4c3abae914fb3fbe3b15 (DEFAULT_TABLE table=default.Table_b8623fa3d1eb479a96e59b16c329e590 [id=c0f00fb75b204087bd476376ddfdc046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.978338 7793 log.cc:1196] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/56d8d5a3ca724141b58e4e2cea23be68
I20260808 13:35:36.982550 5334 tablet_service.cc:1467] Processing CreateTablet for tablet 9d20826f93964ba287c7058c07285d64 (DEFAULT_TABLE table=default.Table_ce81522648e14e5daa8c8fb7d55ef7e4 [id=9df6ef937f0044ab9b0ded9876bead2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.983811 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d20826f93964ba287c7058c07285d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.984140 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 9d20826f93964ba287c7058c07285d64 (DEFAULT_TABLE table=default.Table_ce81522648e14e5daa8c8fb7d55ef7e4 [id=9df6ef937f0044ab9b0ded9876bead2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:36.984643 7793 ts_tablet_manager.cc:1939] T 56d8d5a3ca724141b58e4e2cea23be68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:37.000181 7690 consensus_queue.cc:1035] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:37.015872 5385 raft_consensus.cc:3037] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:37.020579 7690 raft_consensus.cc:482] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:37.021373 7690 raft_consensus.cc:504] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.027979 7690 leader_election.cc:288] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:36.924007 5829 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb821666b9c4a93af438ccba51c0c7c (DEFAULT_TABLE table=default.Table_7e0bc5965bd54d788347d0d41c53208f [id=296621c6dba14fbc8305eca14e4396a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.038255 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb821666b9c4a93af438ccba51c0c7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.931625 5841 tablet_service.cc:1467] Processing CreateTablet for tablet fdacfbbaef114ea7a9da62f8a65916f6 (DEFAULT_TABLE table=default.Table_b84003117fd04b17930f7a3ba4e2d05f [id=7f4fa75c41e542928b94ef51cfd2f33a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.043561 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdacfbbaef114ea7a9da62f8a65916f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.045251 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_4579204171cf47c6b668cedc2efaa649"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:37.046801 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4579204171cf47c6b668cedc2efaa649 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:37.048446 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c06c463992cf45db8dacc3018d8bcb8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:37.050027 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c06c463992cf45db8dacc3018d8bcb8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:37.052052 7690 consensus_queue.cc:1035] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:37.053363 7434 raft_consensus.cc:482] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:37.054081 7434 raft_consensus.cc:504] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.057027 7434 leader_election.cc:288] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:36.969674 7199 ts_tablet_manager.cc:1939] T 2cd6945ecccc4684a3fa5b2a1f07057f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:36.976147 5836 tablet_service.cc:1467] Processing CreateTablet for tablet d5c010e1c8804b7e8e4a58f091cad1d1 (DEFAULT_TABLE table=default.Table_6fc58ab15c1b44c69500f3ad87062bcd [id=562d628bce6b4d96b2013e847c87a8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.067301 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5c010e1c8804b7e8e4a58f091cad1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:36.981559 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 3999ebf74e2e4c3abae914fb3fbe3b15 (DEFAULT_TABLE table=default.Table_b8623fa3d1eb479a96e59b16c329e590 [id=c0f00fb75b204087bd476376ddfdc046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.019718 5833 tablet_service.cc:1467] Processing CreateTablet for tablet 9d20826f93964ba287c7058c07285d64 (DEFAULT_TABLE table=default.Table_ce81522648e14e5daa8c8fb7d55ef7e4 [id=9df6ef937f0044ab9b0ded9876bead2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.027231 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb8c7eeec164485a4e5bb80f9d5fd72" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:37.068667 5883 raft_consensus.cc:2454] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:37.068925 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d20826f93964ba287c7058c07285d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.071357 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3999ebf74e2e4c3abae914fb3fbe3b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.087479 5550 leader_election.cc:302] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:37.087674 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369a942248bd4cec9ee9170fe55601a3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:37.088207 7690 raft_consensus.cc:2781] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:37.088565 7690 raft_consensus.cc:482] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:37.088914 7690 raft_consensus.cc:3037] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:37.089489 5550 leader_election.cc:341] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:37.096992 7690 raft_consensus.cc:504] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.105237 6033 raft_consensus.cc:374] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:37.105527 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb8c7eeec164485a4e5bb80f9d5fd72" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:37.105610 7690 leader_election.cc:288] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:37.106257 5883 raft_consensus.cc:3037] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:37.108934 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:37.110121 4840 catalog_manager.cc:5495] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:37.113791 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:37.115829 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:37.117771 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:37.119027 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:37.120993 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013]). Expected version 6 got 0
I20260808 13:35:37.121673 5883 raft_consensus.cc:2454] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:37.123188 5550 leader_election.cc:302] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:37.105520 6033 raft_consensus.cc:729] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:37.124051 7690 raft_consensus.cc:2781] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:37.124476 7690 raft_consensus.cc:686] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:37.125200 7690 consensus_queue.cc:237] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.173012 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb821666b9c4a93af438ccba51c0c7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.175463 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdacfbbaef114ea7a9da62f8a65916f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.176877 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5c010e1c8804b7e8e4a58f091cad1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.175609 6026 tablet_bootstrap.cc:492] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:37.183120 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3999ebf74e2e4c3abae914fb3fbe3b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.187225 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d20826f93964ba287c7058c07285d64. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:37.198484 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a73e98a8b20144508a5a8d451f4bb302 consensus state has no leader
I20260808 13:35:37.199163 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 56d8d5a3ca724141b58e4e2cea23be68 (table default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212]) successfully deleted
I20260808 13:35:37.205615 6026 tablet_bootstrap.cc:654] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:37.217545 7690 raft_consensus.cc:482] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:37.218086 7690 raft_consensus.cc:504] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.226380 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adf9a03742a14cfe867c578c466838de" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:37.228044 5550 leader_election.cc:341] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:37.154105 6033 consensus_queue.cc:260] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.232550 7690 leader_election.cc:288] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:37.236145 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_702490ba7901458b954c15c4637424b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:37.104856 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 338) took 2136 ms. Trace:
W20260808 13:35:37.237840 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_702490ba7901458b954c15c4637424b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:37.242141 4839 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a8ebe3d6290044f79864293e5c170dd5" } new_table_name: "default.Table_f10f80e85d6c4f87b43c1a53b888716c" modify_external_catalogs: true
I20260808 13:35:37.263199 6026 tablet_bootstrap.cc:492] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:37.263697 6026 ts_tablet_manager.cc:1397] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.088s user 0.008s sys 0.007s
I20260808 13:35:37.267556 7690 consensus_queue.cc:1035] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:37.275280 4841 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:37.276321 4841 catalog_manager.cc:5495] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
W20260808 13:35:37.298439 4839 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8d17615af06b4144a0d7d74c64416451 has no consensus state
I20260808 13:35:37.307905 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_03cf1a01ebc14024a283b7e5ed420074"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:37.309572 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03cf1a01ebc14024a283b7e5ed420074 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:37.287726 6026 raft_consensus.cc:348] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.310431 6026 raft_consensus.cc:374] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:37.310740 6026 raft_consensus.cc:729] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:37.319123 4841 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:37.320353 4841 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:37.320956 6026 consensus_queue.cc:260] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.321635 4841 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:37.327219 7568 raft_consensus.cc:482] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:37.327740 7568 raft_consensus.cc:504] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.323120 4841 catalog_manager.cc:5495] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:37.329769 4841 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013]). Expected version 6 got 0
I20260808 13:35:37.329757 7568 leader_election.cc:288] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:37.322080 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb8c7eeec164485a4e5bb80f9d5fd72" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:37.331351 4841 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:37.334002 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0cc688386f3413ea939dda3a75d6541" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:37.335080 5640 raft_consensus.cc:2379] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:37.330116 6026 ts_tablet_manager.cc:1428] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.066s user 0.006s sys 0.001s
I20260808 13:35:37.382781 6026 tablet_bootstrap.cc:492] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:37.423712 4839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_2b5f163ace454f5e87d04751846435e8" } modify_external_catalogs: true
I20260808 13:35:37.424429 4839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_2b5f163ace454f5e87d04751846435e8" } modify_external_catalogs: true
I20260808 13:35:37.434808 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:37.237515 7199 rpcz_store.cc:276] 0808 13:35:34.968000 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.969241 (+ 1241us) service_pool.cc:227] Handling call
0808 13:35:36.161015 (+1191774us) ts_tablet_manager.cc:1145] Deleting tablet 2cd6945ecccc4684a3fa5b2a1f07057f
0808 13:35:36.161056 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.637118 (+476062us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:36.781838 (+144720us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:37.104795 (+322957us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":114641,"tablet-delete.queue_time_us":1191196}
I20260808 13:35:37.436563 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:37.437256 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 2cd6945ecccc4684a3fa5b2a1f07057f (table default.Table_b75082dac94348fab668160895573637 [id=56f4695998f34301b8528a79b16e2dd2]) successfully deleted
I20260808 13:35:37.437732 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:37.439347 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013]). Expected version 6 got 0
I20260808 13:35:37.440097 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:37.440193 6033 ts_tablet_manager.cc:1428] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.575s user 0.010s sys 0.000s
W20260808 13:35:37.440549 6033 ts_tablet_manager.cc:1460] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Tablet startup took 575ms
I20260808 13:35:37.440908 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
W20260808 13:35:37.440828 6033 ts_tablet_manager.cc:1462] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Trace:
0808 13:35:16.652520 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:16.668415 (+ 15895us) service_pool.cc:227] Handling call
0808 13:35:16.764570 (+ 96155us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:16.764690 (+ 120us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:16.876534 (+111844us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:17.214399 (+337865us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:17.601502 (+387103us) inbound_call.cc:173] Queueing success response
0808 13:35:36.240932 (+18639430us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0808 13:35:36.865301 (+624369us) ts_tablet_manager.cc:1429] Starting tablet replica
0808 13:35:36.912953 (+ 47652us) tablet_replica.cc:259] Starting instrumentation
0808 13:35:36.913703 (+ 750us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":6593,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":286,"tablet-open.queue_time_us":18639571}
I20260808 13:35:37.452325 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0cc688386f3413ea939dda3a75d6541" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:37.454016 6026 tablet_bootstrap.cc:654] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:37.454373 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:37.455370 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:37.456784 4833 catalog_manager.cc:5495] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:37.459276 7560 consensus_queue.cc:1035] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:37.468356 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:37.468905 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:37.479380 4833 catalog_manager.cc:5495] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:37.491837 6026 tablet_bootstrap.cc:492] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:37.492332 6026 ts_tablet_manager.cc:1397] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.110s user 0.013s sys 0.000s
I20260808 13:35:37.498265 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:37.501623 7705 raft_consensus.cc:482] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:37.502681 7705 raft_consensus.cc:504] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.500736 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013]). Expected version 6 got 0
I20260808 13:35:37.503991 6026 raft_consensus.cc:348] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.513569 7705 leader_election.cc:288] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:37.515085 4833 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:37.512832 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66b38e83dab4bf1a3d2d1e088a96988" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:37.515969 5629 raft_consensus.cc:2454] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:37.517347 6026 raft_consensus.cc:374] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:37.517686 6026 raft_consensus.cc:729] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:37.519474 5772 leader_election.cc:302] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:37.519568 6026 consensus_queue.cc:260] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.529940 7560 raft_consensus.cc:2781] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:37.530916 7560 raft_consensus.cc:482] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:37.531561 7560 raft_consensus.cc:3037] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:37.539726 6026 ts_tablet_manager.cc:1428] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.042s user 0.001s sys 0.005s
I20260808 13:35:37.544427 5379 raft_consensus.cc:3037] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:37.556499 7199 tablet_replica.cc:330] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:37.574892 6026 tablet_bootstrap.cc:492] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:37.578933 7199 raft_consensus.cc:2229] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:37.579702 7199 raft_consensus.cc:2258] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:37.581777 7199 ts_tablet_manager.cc:1905] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:37.619946 6026 tablet_bootstrap.cc:654] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:37.634894 5283 leader_election.cc:302] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:37.643481 7199 ts_tablet_manager.cc:1918] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:37.644022 7199 log.cc:1196] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/1f9bc338093f403881d329a7e913c9f7
I20260808 13:35:37.644882 7199 ts_tablet_manager.cc:1939] T 1f9bc338093f403881d329a7e913c9f7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
W20260808 13:35:37.662691 6450 thread.cc:636] rpc worker (service pool kudu.server.GenericService) Time spent creating pthread: real 0.726s user 0.158s sys 0.300s
W20260808 13:35:37.663014 6450 thread.cc:602] rpc worker (service pool kudu.server.GenericService) Time spent starting thread: real 0.726s user 0.158s sys 0.300s
I20260808 13:35:37.670585 6033 tablet_bootstrap.cc:492] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:37.672626 6026 tablet_bootstrap.cc:492] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:37.673058 6026 ts_tablet_manager.cc:1397] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.132s user 0.010s sys 0.004s
W20260808 13:35:37.674237 5385 consensus_meta.cc:287] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c: Time spent flushing consensus metadata: real 0.657s user 0.001s sys 0.010s
I20260808 13:35:37.675226 6026 raft_consensus.cc:348] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.675866 6026 raft_consensus.cc:374] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:37.676100 6026 raft_consensus.cc:729] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:37.676714 6026 consensus_queue.cc:260] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.678539 6026 ts_tablet_manager.cc:1428] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260808 13:35:37.684111 7844 raft_consensus.cc:482] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:37.684660 7844 raft_consensus.cc:504] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.687662 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b07b7d31ef41e484389c0f007598f0" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:37.689327 5550 leader_election.cc:341] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:37.689899 7844 leader_election.cc:288] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:37.698522 4823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.60:37135: connect: Connection refused (error 111) [suppressed 3 similar messages]
I20260808 13:35:37.699470 7844 raft_consensus.cc:482] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:37.700565 7844 raft_consensus.cc:504] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.704711 7560 raft_consensus.cc:504] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.708011 7844 leader_election.cc:288] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:37.449779 5356 consensus_meta.cc:287] T 0cb821666b9c4a93af438ccba51c0c7c P 946c9d8462644deba5b45fa724f1923c: Time spent flushing consensus metadata: real 0.547s user 0.008s sys 0.000s
W20260808 13:35:37.666368 5357 consensus_meta.cc:287] T d5c010e1c8804b7e8e4a58f091cad1d1 P 946c9d8462644deba5b45fa724f1923c: Time spent flushing consensus metadata: real 0.706s user 0.007s sys 0.000s
W20260808 13:35:37.679720 5351 consensus_meta.cc:287] T fdacfbbaef114ea7a9da62f8a65916f6 P 946c9d8462644deba5b45fa724f1923c: Time spent flushing consensus metadata: real 0.704s user 0.007s sys 0.001s
W20260808 13:35:37.735742 7843 log.cc:924] Time spent T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260808 13:35:37.742714 6046 tablet_bootstrap.cc:654] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:37.683723 7013 ts_tablet_manager.cc:1918] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260808 13:35:37.723451 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 46540.6us
I20260808 13:35:37.758662 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66b38e83dab4bf1a3d2d1e088a96988" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:37.765856 7013 log.cc:1196] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/896389941179456d9d0ed3cd14726f11
I20260808 13:35:37.775621 7830 raft_consensus.cc:2726] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:37.779814 7013 ts_tablet_manager.cc:1939] T 896389941179456d9d0ed3cd14726f11 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:37.781003 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 317) took 5724 ms. Trace:
I20260808 13:35:37.781363 7013 rpcz_store.cc:276] 0808 13:35:32.056581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.056719 (+ 138us) service_pool.cc:227] Handling call
0808 13:35:36.423151 (+4366432us) ts_tablet_manager.cc:1145] Deleting tablet 896389941179456d9d0ed3cd14726f11
0808 13:35:36.423465 (+ 314us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:36.847700 (+424235us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:37.683494 (+835794us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:37.780936 (+ 97442us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":38337,"tablet-delete.queue_time_us":4365133}
I20260808 13:35:37.755340 6026 tablet_bootstrap.cc:492] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:37.794039 7716 raft_consensus.cc:482] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5cd987c3dbdb4168beb73cd7e577c7cb)
I20260808 13:35:37.794827 7716 raft_consensus.cc:504] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:37.809340 6026 tablet_bootstrap.cc:654] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:37.814798 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7284afe0294644a368cf51451c7ab6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:37.820591 7560 leader_election.cc:288] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:37.822803 5608 tablet_service.cc:1467] Processing CreateTablet for tablet d4497e1f59c04ad190382259705b1b54 (DEFAULT_TABLE table=default.Table_0913077ace074f11bf8a6222338489c6 [id=e048189f6ef441ce8e17c60f0f5b0cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.822968 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 0d8357611ced42db91bd1e416f0e26b3 (DEFAULT_TABLE table=default.Table_a446b3e7bb7944b0a5b119dd6a1b6d7c [id=4bfd4a2482364643a6d15f7a638f2eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.823733 6033 tablet_bootstrap.cc:654] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:37.823808 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4497e1f59c04ad190382259705b1b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.824690 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66b38e83dab4bf1a3d2d1e088a96988" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:37.825282 5629 raft_consensus.cc:3037] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:37.705996 5833 consensus_meta.cc:287] T 9d20826f93964ba287c7058c07285d64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.604s user 0.008s sys 0.000s
W20260808 13:35:37.706446 5836 consensus_meta.cc:287] T d5c010e1c8804b7e8e4a58f091cad1d1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.607s user 0.009s sys 0.000s
I20260808 13:35:37.707419 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:37.708528 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 339) took 2739 ms. Trace:
I20260808 13:35:37.829597 7199 rpcz_store.cc:276] 0808 13:35:34.968737 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.971406 (+ 2669us) service_pool.cc:227] Handling call
0808 13:35:37.438619 (+2467213us) ts_tablet_manager.cc:1145] Deleting tablet 1f9bc338093f403881d329a7e913c9f7
0808 13:35:37.556313 (+117694us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:37.635040 (+ 78727us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:37.643241 (+ 8201us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:37.708468 (+ 65227us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20909,"tablet-delete.queue_time_us":2466513}
I20260808 13:35:37.830852 7199 tablet_replica.cc:330] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:37.831629 7199 raft_consensus.cc:2229] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:37.832208 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 1f9bc338093f403881d329a7e913c9f7 (table default.Table_49299116c37c4b9dbb509dbc8efae85e [id=d88dc1106b9b4e1eb18dc680d15fdfcf]) successfully deleted
I20260808 13:35:37.832427 7199 raft_consensus.cc:2258] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:37.721442 5841 consensus_meta.cc:287] T fdacfbbaef114ea7a9da62f8a65916f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.656s user 0.010s sys 0.001s
W20260808 13:35:37.835009 5550 leader_election.cc:341] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:37.834394 7199 ts_tablet_manager.cc:1905] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:37.836031 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d8357611ced42db91bd1e416f0e26b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.834067 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 0d8357611ced42db91bd1e416f0e26b3 (DEFAULT_TABLE table=default.Table_a446b3e7bb7944b0a5b119dd6a1b6d7c [id=4bfd4a2482364643a6d15f7a638f2eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.839295 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d8357611ced42db91bd1e416f0e26b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.844740 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 3024717477814906a3d861df35e4cb6c (DEFAULT_TABLE table=default.Table_e4788552c1a8457a8d34951f6827c5e4 [id=a14f97b5af3741bebd7d88f5602675f1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:37.730146 5842 consensus_meta.cc:287] T 3999ebf74e2e4c3abae914fb3fbe3b15 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.632s user 0.010s sys 0.001s
I20260808 13:35:37.845854 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3024717477814906a3d861df35e4cb6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.811779 5832 tablet_service.cc:1467] Processing CreateTablet for tablet d4497e1f59c04ad190382259705b1b54 (DEFAULT_TABLE table=default.Table_0913077ace074f11bf8a6222338489c6 [id=e048189f6ef441ce8e17c60f0f5b0cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.847664 5832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4497e1f59c04ad190382259705b1b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.861109 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 3024717477814906a3d861df35e4cb6c (DEFAULT_TABLE table=default.Table_e4788552c1a8457a8d34951f6827c5e4 [id=a14f97b5af3741bebd7d88f5602675f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.861997 5598 tablet_service.cc:1467] Processing CreateTablet for tablet 8d17615af06b4144a0d7d74c64416451 (DEFAULT_TABLE table=default.Table_f10f80e85d6c4f87b43c1a53b888716c [id=20c27e5e63df49a8bcf6faef31940880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.863104 5598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d17615af06b4144a0d7d74c64416451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.862430 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3024717477814906a3d861df35e4cb6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.864363 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 8d17615af06b4144a0d7d74c64416451 (DEFAULT_TABLE table=default.Table_f10f80e85d6c4f87b43c1a53b888716c [id=20c27e5e63df49a8bcf6faef31940880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.865340 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d17615af06b4144a0d7d74c64416451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.867337 5603 tablet_service.cc:1467] Processing CreateTablet for tablet ef2b958e376140f7949aa04d49a4aec6 (DEFAULT_TABLE table=default.Table_82d49330ca914820a6cb8b3591280a90 [id=dc1f8a5aa37a4ad99be0715831a680fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.868250 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef2b958e376140f7949aa04d49a4aec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.872788 5833 tablet_service.cc:1467] Processing CreateTablet for tablet ef2b958e376140f7949aa04d49a4aec6 (DEFAULT_TABLE table=default.Table_82d49330ca914820a6cb8b3591280a90 [id=dc1f8a5aa37a4ad99be0715831a680fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.872952 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:37.873786 5833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef2b958e376140f7949aa04d49a4aec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.874960 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:37.875303 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 8e843a211ab24792b9a475cf08e8c49b (DEFAULT_TABLE table=default.Table_8a69b0e5577e4c6f9c76504751d2c1a4 [id=1b8762a3419f47bb9c060294f70d62aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.875967 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:37.876508 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e843a211ab24792b9a475cf08e8c49b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.876456 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:37.887106 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 318) took 5829 ms. Trace:
I20260808 13:35:37.887490 7013 rpcz_store.cc:276] 0808 13:35:32.058189 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.058334 (+ 145us) service_pool.cc:227] Handling call
0808 13:35:37.782475 (+5724141us) ts_tablet_manager.cc:1145] Deleting tablet 024e7f2daaad4f48a79b94344ff35e98
0808 13:35:37.782523 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:37.887036 (+104513us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5722825}
I20260808 13:35:37.899514 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 64be4bede9d04d2db0940a75e0e9b2fb (DEFAULT_TABLE table=default.Table_6b9bc590a69746909348fbcbb7856cdf [id=03f2c8cadea14860aee2cd51574379b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.900720 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64be4bede9d04d2db0940a75e0e9b2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.903323 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 328d361c336848689c6906ca6fe2ccc3
I20260808 13:35:37.903503 5829 tablet_service.cc:1467] Processing CreateTablet for tablet 8e843a211ab24792b9a475cf08e8c49b (DEFAULT_TABLE table=default.Table_8a69b0e5577e4c6f9c76504751d2c1a4 [id=1b8762a3419f47bb9c060294f70d62aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.904819 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e843a211ab24792b9a475cf08e8c49b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:37.904837 4842 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 399) took 2790 ms. Trace:
I20260808 13:35:37.912346 5826 tablet_service.cc:1514] Processing DeleteTablet for tablet 328d361c336848689c6906ca6fe2ccc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:35 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:37.905144 4842 rpcz_store.cc:276] 0808 13:35:35.114596 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.888673 (+774077us) service_pool.cc:227] Handling call
0808 13:35:35.922850 (+ 34177us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff466 00007f227bd40b34 00007f227bd40ac2 00007f227bd409f1 00007f227bd4096c 00007f227bd3fc22 00007f2279fde94c 00007f2279fddd86
0808 13:35:35.924459 (+ 1609us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:35.925020 (+ 561us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:35.925141 (+ 121us) catalog_manager.cc:2761] Locking tablets
0808 13:35:37.071254 (+1146113us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:37.148567 (+ 77313us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:37.159665 (+ 11098us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:37.903204 (+743539us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:37.904772 (+ 1568us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:37.079509 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:37.079845 (+ 336us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:37.079867 (+ 22us) write_op.cc:435] Acquired schema lock
0808 13:35:37.079880 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:37.080178 (+ 298us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:37.080217 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:37.080238 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:37.080480 (+ 242us) tablet.cc:699] Row locks acquired
0808 13:35:37.080494 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:37.080591 (+ 97us) write_op.cc:270] Start()
0808 13:35:37.080663 (+ 72us) write_op.cc:276] Timestamp: P: 1786196137080547 usec, L: 0
0808 13:35:37.080679 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:37.081141 (+ 462us) log.cc:842] Serialized 759 byte log entry
0808 13:35:37.141287 (+ 60146us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b58000c05b0. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c1a9834 00007f227c1ae771 00007f227c1ae6f9 00007f227c1ae689 00007f227c1ae651 00007f227c1ad81d 0000000000580784 00000000005805b9 00007f22752da6b6 00007f22752ddb11
0808 13:35:37.145519 (+ 4232us) op_driver.cc:444] REPLICATION: finished
0808 13:35:37.146156 (+ 637us) write_op.cc:301] APPLY: starting
0808 13:35:37.146220 (+ 64us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:37.146365 (+ 145us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:37.146395 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:37.146812 (+ 417us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:37.147089 (+ 277us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:37.147146 (+ 57us) write_op.cc:312] APPLY: finished
0808 13:35:37.147451 (+ 305us) log.cc:842] Serialized 29 byte log entry
0808 13:35:37.147771 (+ 320us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.147894 (+ 123us) write_op.cc:454] Released schema lock
0808 13:35:37.147954 (+ 60us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:37.148181 (+ 227us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.148214 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":27402624,"child_traces":[["op",{"apply.queue_time_us":327,"apply.run_cpu_time_us":2305,"apply.run_wall_time_us":2385,"mutex_wait_us":4605,"num_ops":2,"prepare.queue_time_us":6402,"prepare.run_cpu_time_us":1870,"prepare.run_wall_time_us":1871,"raft.queue_time_us":1307,"raft.run_cpu_time_us":1220,"raft.run_wall_time_us":58685,"replication_time_us":64772,"spinlock_wait_cycles":133738752}]]}
I20260808 13:35:37.950127 7879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.60:37135 every 8 connection(s)
I20260808 13:35:37.966080 6450 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.60:37135
I20260808 13:35:37.977932 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:37.984659 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:37.985919 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 64be4bede9d04d2db0940a75e0e9b2fb (DEFAULT_TABLE table=default.Table_6b9bc590a69746909348fbcbb7856cdf [id=03f2c8cadea14860aee2cd51574379b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:37.990103 7199 ts_tablet_manager.cc:1918] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:37.999626 5826 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3ad9ca75b541798bc04d7701128faa (DEFAULT_TABLE table=default.Table_642fc6953dc8471cbd814ae6fda0cc6a [id=9136fce84d3d447a999d24131713ce4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.001344 7199 log.cc:1196] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/127e8d0323954ef788018bcb91582951
I20260808 13:35:38.002524 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:38.004019 7199 ts_tablet_manager.cc:1939] T 127e8d0323954ef788018bcb91582951 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:38.008725 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:38.011858 7716 leader_election.cc:288] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:38.021000 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 314) took 5969 ms. Trace:
I20260808 13:35:38.021651 7013 rpcz_store.cc:276] 0808 13:35:32.051889 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.052022 (+ 133us) service_pool.cc:227] Handling call
0808 13:35:37.999595 (+5947573us) ts_tablet_manager.cc:1145] Deleting tablet 06c59eea10ec45af8e3a0992f9cc74bf
0808 13:35:38.020652 (+ 21057us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b5800001898. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c2091af11 00007f5c20a9e851 00007f5c20a9ee82 00007f5c2995e8b9 00007f5c289925d8 00007f5c28a56389 00007f5c28a56983 00007f5c28a86e10 00007f5c28a6de0d 00007f5c28a6ddb9 00007f5c28a6dd49
0808 13:35:38.020705 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.020939 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13887872,"tablet-delete.queue_time_us":5828325}
I20260808 13:35:38.023303 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 319) took 5894 ms. Trace:
I20260808 13:35:38.023679 7013 rpcz_store.cc:276] 0808 13:35:32.128660 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.129117 (+ 457us) service_pool.cc:227] Handling call
0808 13:35:38.022896 (+5893779us) ts_tablet_manager.cc:1145] Deleting tablet 8d12106f0d794a79935999a2b8dbe7e9
0808 13:35:38.022956 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.023255 (+ 299us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5893156}
I20260808 13:35:38.025012 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 320) took 5896 ms. Trace:
I20260808 13:35:38.025406 7013 rpcz_store.cc:276] 0808 13:35:32.128982 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.129920 (+ 938us) service_pool.cc:227] Handling call
0808 13:35:38.024594 (+5894674us) ts_tablet_manager.cc:1145] Deleting tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be
0808 13:35:38.024656 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.024960 (+ 304us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5894082}
I20260808 13:35:38.026706 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 321) took 5701 ms. Trace:
I20260808 13:35:38.027878 7013 rpcz_store.cc:276] 0808 13:35:32.325386 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:32.325542 (+ 156us) service_pool.cc:227] Handling call
0808 13:35:38.026292 (+5700750us) ts_tablet_manager.cc:1145] Deleting tablet 20113491b108476b95b8f32b69176d0e
0808 13:35:38.026351 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.026646 (+ 295us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5688438}
I20260808 13:35:38.029467 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 322) took 4207 ms. Trace:
I20260808 13:35:38.029942 7013 rpcz_store.cc:276] 0808 13:35:33.822663 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.822921 (+ 258us) service_pool.cc:227] Handling call
0808 13:35:38.029044 (+4206123us) ts_tablet_manager.cc:1145] Deleting tablet 1ba5fa465ec04bbb801a821a71a8f87d
0808 13:35:38.029097 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.029410 (+ 313us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4205266}
I20260808 13:35:38.035637 4838 catalog_manager.cc:5495] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:38.040493 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 14a9373000e24e75858e6d7457587264 (DEFAULT_TABLE table=default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.043892 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:38.046516 6450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
I20260808 13:35:38.050230 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64be4bede9d04d2db0940a75e0e9b2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.055449 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 6450
I20260808 13:35:38.062103 5826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3ad9ca75b541798bc04d7701128faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.066077 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 341) took 3095 ms. Trace:
I20260808 13:35:38.067159 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a9373000e24e75858e6d7457587264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.066490 7199 rpcz_store.cc:276] 0808 13:35:34.971059 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.971166 (+ 107us) service_pool.cc:227] Handling call
0808 13:35:37.830575 (+2859409us) ts_tablet_manager.cc:1145] Deleting tablet 127e8d0323954ef788018bcb91582951
0808 13:35:37.830633 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:37.900922 (+ 70289us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:37.989882 (+ 88960us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.065999 (+ 76117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2858324}
I20260808 13:35:38.078374 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013]). Expected version 6 got 0
I20260808 13:35:38.081086 7199 tablet_replica.cc:330] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:38.083321 7199 raft_consensus.cc:2229] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:38.079493 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 127e8d0323954ef788018bcb91582951 (table default.Table_23396b27c08444c7981e55a66139f6b6 [id=2004b79d44ab440ba8d0004db91b6bce]) successfully deleted
I20260808 13:35:38.087334 7199 raft_consensus.cc:2258] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:38.090574 7199 ts_tablet_manager.cc:1905] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:38.092079 4837 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 402) took 2120 ms. Trace:
I20260808 13:35:38.032095 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369a942248bd4cec9ee9170fe55601a3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:38.033988 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb8c7eeec164485a4e5bb80f9d5fd72" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:38.035356 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adf9a03742a14cfe867c578c466838de" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:38.037477 5385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.170.2:56477 (request call id 204) took 1044 ms. Trace:
I20260808 13:35:38.095798 5385 rpcz_store.cc:276] 0808 13:35:36.992614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:37.014916 (+ 22302us) service_pool.cc:227] Handling call
0808 13:35:37.015494 (+ 578us) raft_consensus.cc:1457] Updating replica for 1 ops
0808 13:35:37.750699 (+735205us) raft_consensus.cc:1520] Early marking committed up to index 0
0808 13:35:37.750746 (+ 47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0808 13:35:37.751212 (+ 466us) log.cc:842] Serialized 26 byte log entry
0808 13:35:37.792838 (+ 41626us) raft_consensus.cc:1631] Marking committed up to 1
0808 13:35:37.793381 (+ 543us) log.cc:842] Serialized 14 byte log entry
0808 13:35:37.793699 (+ 318us) raft_consensus.cc:1683] Filling consensus response to leader.
0808 13:35:37.793829 (+ 130us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0808 13:35:38.037250 (+243421us) raft_consensus.cc:1671] finished
0808 13:35:38.037274 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0808 13:35:38.037408 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":176,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":9390,"raft.run_cpu_time_us":530,"raft.run_wall_time_us":540,"thread_start_us":41126,"threads_started":1,"wal-append.queue_time_us":280225}
I20260808 13:35:38.038797 6046 tablet_bootstrap.cc:492] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:38.089109 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b07b7d31ef41e484389c0f007598f0" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:38.090195 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:38.106775 6046 ts_tablet_manager.cc:1397] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 1.190s user 0.016s sys 0.010s
I20260808 13:35:38.092420 4837 rpcz_store.cc:276] 0808 13:35:35.971962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.975036 (+ 3074us) service_pool.cc:227] Handling call
0808 13:35:36.115104 (+140068us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff7d6 00007f227bd45634 00007f227bd455c2 00007f227bd454f1 00007f227bd4546c 00007f227bd44722 00007f2279fde94c 00007f2279fddd86
0808 13:35:36.115987 (+ 883us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:36.116709 (+ 722us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:36.117286 (+ 577us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:36.156337 (+ 39051us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:36.267404 (+111067us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:38.092017 (+1824613us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:36.176812 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:36.177161 (+ 349us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:36.177182 (+ 21us) write_op.cc:435] Acquired schema lock
0808 13:35:36.177194 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:36.177445 (+ 251us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:36.177479 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:36.177501 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:36.255103 (+ 77602us) tablet.cc:699] Row locks acquired
0808 13:35:36.255120 (+ 17us) write_op.cc:260] PREPARE: finished
0808 13:35:36.255222 (+ 102us) write_op.cc:270] Start()
0808 13:35:36.255315 (+ 93us) write_op.cc:276] Timestamp: P: 1786196136255151 usec, L: 0
0808 13:35:36.255331 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:36.255979 (+ 648us) log.cc:842] Serialized 405 byte log entry
0808 13:35:36.264542 (+ 8563us) op_driver.cc:444] REPLICATION: finished
0808 13:35:36.265072 (+ 530us) write_op.cc:301] APPLY: starting
0808 13:35:36.265146 (+ 74us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:36.265257 (+ 111us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:36.265483 (+ 226us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:36.265667 (+ 184us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:36.265848 (+ 181us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:36.265908 (+ 60us) write_op.cc:312] APPLY: finished
0808 13:35:36.266110 (+ 202us) log.cc:842] Serialized 23 byte log entry
0808 13:35:36.266717 (+ 607us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:36.266800 (+ 83us) write_op.cc:454] Released schema lock
0808 13:35:36.266831 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:36.267041 (+ 210us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:36.267072 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":320264960,"child_traces":[["op",{"apply.queue_time_us":355,"apply.run_cpu_time_us":1670,"apply.run_wall_time_us":2157,"num_ops":1,"prepare.queue_time_us":19376,"prepare.run_cpu_time_us":1916,"prepare.run_wall_time_us":109559,"raft.queue_time_us":78,"raft.run_cpu_time_us":852,"raft.run_wall_time_us":1642,"replication_time_us":9135}]]}
I20260808 13:35:38.112375 6046 raft_consensus.cc:348] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.113078 6046 raft_consensus.cc:374] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:38.113348 6046 raft_consensus.cc:729] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:38.114343 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 323) took 4291 ms. Trace:
I20260808 13:35:38.114667 7013 rpcz_store.cc:276] 0808 13:35:33.823117 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.823318 (+ 201us) service_pool.cc:227] Handling call
0808 13:35:38.113995 (+4290677us) ts_tablet_manager.cc:1145] Deleting tablet 3451d72548a140c68a6ccc2b566a09b1
0808 13:35:38.114035 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.114290 (+ 255us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4290019}
W20260808 13:35:38.120904 4840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 38) took 2984 ms (client timeout 1000 ms). Trace:
W20260808 13:35:38.121325 4840 rpcz_store.cc:269] 0808 13:35:35.114187 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.349722 (+235535us) service_pool.cc:227] Handling call
0808 13:35:38.098560 (+2748838us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:37.235606 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:37.235983 (+ 377us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:37.236009 (+ 26us) write_op.cc:435] Acquired schema lock
0808 13:35:37.236021 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:37.236289 (+ 268us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:37.236324 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:37.236345 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:37.236506 (+ 161us) tablet.cc:699] Row locks acquired
0808 13:35:37.236519 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:37.236615 (+ 96us) write_op.cc:270] Start()
0808 13:35:37.236695 (+ 80us) write_op.cc:276] Timestamp: P: 1786196137236553 usec, L: 0
0808 13:35:37.236710 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:37.237217 (+ 507us) log.cc:842] Serialized 527 byte log entry
0808 13:35:37.256005 (+ 18788us) op_driver.cc:444] REPLICATION: finished
0808 13:35:37.256286 (+ 281us) write_op.cc:301] APPLY: starting
0808 13:35:37.256338 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:37.256422 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:37.256512 (+ 90us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:37.256693 (+ 181us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:37.256828 (+ 135us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:37.256873 (+ 45us) write_op.cc:312] APPLY: finished
0808 13:35:37.257052 (+ 179us) log.cc:842] Serialized 23 byte log entry
0808 13:35:37.257210 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.257281 (+ 71us) write_op.cc:454] Released schema lock
0808 13:35:37.257317 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:37.257517 (+ 200us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.266712 (+ 9195us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":142911,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":142,"apply.run_cpu_time_us":1573,"apply.run_wall_time_us":10719,"num_ops":1,"prepare.queue_time_us":108387,"prepare.run_cpu_time_us":1890,"prepare.run_wall_time_us":7506,"raft.queue_time_us":4189,"raft.run_cpu_time_us":575,"raft.run_wall_time_us":575,"replication_time_us":19214}]]}
I20260808 13:35:38.126240 4840 master_service.cc:946] ReplaceTablet: received request to replace tablet 7e8cf06462484c99a860762a82f7ad64 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:38.128304 6046 consensus_queue.cc:260] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:38.135568 4841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 48) took 2654 ms (client timeout 1000 ms). Trace:
W20260808 13:35:38.136044 4841 rpcz_store.cc:269] 0808 13:35:35.481335 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.574767 (+ 93432us) service_pool.cc:227] Handling call
0808 13:35:38.135486 (+2560719us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:37.335298 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:37.335784 (+ 486us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:37.335810 (+ 26us) write_op.cc:435] Acquired schema lock
0808 13:35:37.335826 (+ 16us) tablet.cc:660] Decoding operations
0808 13:35:37.336166 (+ 340us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:37.336204 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:37.336227 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:37.336654 (+ 427us) tablet.cc:699] Row locks acquired
0808 13:35:37.336672 (+ 18us) write_op.cc:260] PREPARE: finished
0808 13:35:37.336814 (+ 142us) write_op.cc:270] Start()
0808 13:35:37.336929 (+ 115us) write_op.cc:276] Timestamp: P: 1786196137336734 usec, L: 0
0808 13:35:37.336951 (+ 22us) op_driver.cc:331] REPLICATION: starting
0808 13:35:37.337461 (+ 510us) log.cc:842] Serialized 901 byte log entry
0808 13:35:37.406266 (+ 68805us) op_driver.cc:444] REPLICATION: finished
0808 13:35:37.409852 (+ 3586us) write_op.cc:301] APPLY: starting
0808 13:35:37.409908 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:37.410177 (+ 269us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:37.410368 (+ 191us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:37.410930 (+ 562us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:37.411178 (+ 248us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:37.411250 (+ 72us) write_op.cc:312] APPLY: finished
0808 13:35:37.411554 (+ 304us) log.cc:842] Serialized 29 byte log entry
0808 13:35:37.411824 (+ 270us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.411960 (+ 136us) write_op.cc:454] Released schema lock
0808 13:35:37.412030 (+ 70us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:37.412427 (+ 397us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.412495 (+ 68us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35490,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":3334,"apply.run_cpu_time_us":3025,"apply.run_wall_time_us":3025,"num_ops":2,"prepare.queue_time_us":117,"prepare.run_cpu_time_us":2392,"prepare.run_wall_time_us":2392,"replication_time_us":69185}]]}
I20260808 13:35:38.115721 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 327) took 4291 ms. Trace:
I20260808 13:35:38.143380 7013 rpcz_store.cc:276] 0808 13:35:33.824453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:33.825611 (+ 1158us) service_pool.cc:227] Handling call
0808 13:35:38.115424 (+4289813us) ts_tablet_manager.cc:1145] Deleting tablet 61dfd47db0a74ed8b7228b559dd68459
0808 13:35:38.115458 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.115674 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4289165}
I20260808 13:35:38.097703 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7284afe0294644a368cf51451c7ab6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:38.105675 6033 tablet_bootstrap.cc:492] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:38.108345 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9bc338093f403881d329a7e913c9f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:20 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:38.108593 5831 tablet_service.cc:1514] Processing DeleteTablet for tablet 31fc9fdb5ef543da833f9af6ca5807a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 67327badb2524a2c817045a893bcad6e) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:38.145541 5283 leader_election.cc:302] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:38.108817 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:25 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:38.109760 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 7b128feff212456faa1769efd96f6bd8 (DEFAULT_TABLE table=default.Table_034b94572e274eff86802d9cae6c907d [id=087eec2212324876bebfc252a552d074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.145116 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 340) took 4087 ms. Trace:
I20260808 13:35:38.146226 7013 rpcz_store.cc:276] 0808 13:35:34.057304 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.057482 (+ 178us) service_pool.cc:227] Handling call
0808 13:35:38.144719 (+4087237us) ts_tablet_manager.cc:1145] Deleting tablet cb7f6487e28f4aaa94d448be71f8a3c2
0808 13:35:38.144777 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.145064 (+ 287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4086512}
I20260808 13:35:38.146710 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b128feff212456faa1769efd96f6bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.147353 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 342) took 4086 ms. Trace:
I20260808 13:35:38.147611 7013 rpcz_store.cc:276] 0808 13:35:34.060537 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.060724 (+ 187us) service_pool.cc:227] Handling call
0808 13:35:38.147021 (+4086297us) ts_tablet_manager.cc:1145] Deleting tablet 293e9540498e420d8ec873472d0d29fc
0808 13:35:38.147059 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.147305 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4085660}
I20260808 13:35:38.148693 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 349) took 3309 ms. Trace:
I20260808 13:35:38.110416 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 127e8d0323954ef788018bcb91582951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:38.148967 7013 rpcz_store.cc:276] 0808 13:35:34.839139 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.839377 (+ 238us) service_pool.cc:227] Handling call
0808 13:35:38.148370 (+3308993us) ts_tablet_manager.cc:1145] Deleting tablet 1f0a6d9bd2c74d0790badd7977f027e3
0808 13:35:38.148405 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.148645 (+ 240us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3270575}
I20260808 13:35:38.150163 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 350) took 3297 ms. Trace:
I20260808 13:35:38.154968 6033 ts_tablet_manager.cc:1397] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.485s user 0.014s sys 0.006s
I20260808 13:35:38.157748 7716 raft_consensus.cc:2726] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:38.161370 6046 ts_tablet_manager.cc:1428] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.054s user 0.005s sys 0.000s
I20260808 13:35:38.162463 6046 tablet_bootstrap.cc:492] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:38.175524 5771 leader_election.cc:341] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:38.178537 7705 consensus_queue.cc:1035] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:38.179491 6033 raft_consensus.cc:348] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:38.183796 5771 consensus_peers.cc:487] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:38.186087 6033 raft_consensus.cc:374] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:38.182175 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 896389941179456d9d0ed3cd14726f11 (table default.Table_41272f24d0a5431b9f718128ad86cb81 [id=d7c7a7f08fa543aaa0bca2e996d1ee8e]) successfully deleted
I20260808 13:35:38.191483 6033 raft_consensus.cc:729] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
W20260808 13:35:38.191821 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 024e7f2daaad4f48a79b94344ff35e98 because the tablet was not found. No further retry: Not found: Tablet not found: 024e7f2daaad4f48a79b94344ff35e98
W20260808 13:35:38.192446 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 06c59eea10ec45af8e3a0992f9cc74bf because the tablet was not found. No further retry: Not found: Tablet not found: 06c59eea10ec45af8e3a0992f9cc74bf
W20260808 13:35:38.193017 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 8d12106f0d794a79935999a2b8dbe7e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d12106f0d794a79935999a2b8dbe7e9 already in progress: creating tablet
W20260808 13:35:38.193605 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be already in progress: creating tablet
W20260808 13:35:38.194217 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 20113491b108476b95b8f32b69176d0e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 20113491b108476b95b8f32b69176d0e already in progress: creating tablet
I20260808 13:35:38.193274 6033 consensus_queue.cc:260] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.192569 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_465ac7f2a60c415a8809fdfccb4b5e27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:38.205797 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_465ac7f2a60c415a8809fdfccb4b5e27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:38.211048 6033 ts_tablet_manager.cc:1428] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.056s user 0.005s sys 0.001s
I20260808 13:35:38.214289 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 7e8cf06462484c99a860762a82f7ad64
I20260808 13:35:38.216241 4840 catalog_manager.cc:6565] ReplaceTablet: tablet 7e8cf06462484c99a860762a82f7ad64 deleted and replaced by tablet 16736500f94c43d3a3163f61b7606aad
I20260808 13:35:38.220275 4840 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60270 (request call id 417) took 1065 ms. Trace:
I20260808 13:35:38.222752 5948 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:35:38.220839 4840 rpcz_store.cc:276] 0808 13:35:37.154488 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.126066 (+971578us) service_pool.cc:227] Handling call
0808 13:35:38.220220 (+ 94154us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:38.183401 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:38.183766 (+ 365us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:38.183789 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:38.183802 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:38.184098 (+ 296us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:38.184137 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:38.184159 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:38.184339 (+ 180us) tablet.cc:699] Row locks acquired
0808 13:35:38.184362 (+ 23us) write_op.cc:260] PREPARE: finished
0808 13:35:38.184465 (+ 103us) write_op.cc:270] Start()
0808 13:35:38.184541 (+ 76us) write_op.cc:276] Timestamp: P: 1786196138184409 usec, L: 0
0808 13:35:38.184557 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:38.186876 (+ 2319us) log.cc:842] Serialized 622 byte log entry
0808 13:35:38.202295 (+ 15419us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b58000c05b0. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c1a9834 00007f227c1ae771 00007f227c1ae6f9 00007f227c1ae689 00007f227c1ae651 00007f227c1ad81d 0000000000580784 00000000005805b9 00007f22752da6b6 00007f22752ddb11
0808 13:35:38.203615 (+ 1320us) op_driver.cc:444] REPLICATION: finished
0808 13:35:38.204270 (+ 655us) write_op.cc:301] APPLY: starting
0808 13:35:38.204327 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:38.204431 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:38.204451 (+ 20us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:38.211431 (+ 6980us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:38.211642 (+ 211us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:38.211686 (+ 44us) write_op.cc:312] APPLY: finished
0808 13:35:38.211898 (+ 212us) log.cc:842] Serialized 29 byte log entry
0808 13:35:38.212048 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.212117 (+ 69us) write_op.cc:454] Released schema lock
0808 13:35:38.212152 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:38.212376 (+ 224us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.212412 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":505,"apply.run_cpu_time_us":7060,"apply.run_wall_time_us":8330,"num_ops":2,"prepare.queue_time_us":52718,"prepare.run_cpu_time_us":2007,"prepare.run_wall_time_us":3811,"raft.queue_time_us":3321,"raft.run_cpu_time_us":1896,"raft.run_wall_time_us":14605,"replication_time_us":18986,"spinlock_wait_cycles":30414080}]]}
I20260808 13:35:38.223606 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8cf06462484c99a860762a82f7ad64 with delete_type TABLET_DATA_DELETED (replaced by tablet 16736500f94c43d3a3163f61b7606aad) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:38.238934 5462 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:35:38.231658 6033 tablet_bootstrap.cc:492] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:38.255761 4821 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 538d9a883e9d46668fde30876352d20e (table default.Table_41f3478fe8c94fbfaf7fcbdee22ce987 [id=e3e121c36ff44ea0a3a98185c8cce381], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 538d9a883e9d46668fde30876352d20e consensus state has no leader
I20260808 13:35:38.264772 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d1a5d56bb11c4d0a87b86c1b82f4180f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:38.266647 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1a5d56bb11c4d0a87b86c1b82f4180f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:38.281549 5629 raft_consensus.cc:2454] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:38.289119 5357 tablet_service.cc:1467] Processing CreateTablet for tablet d4497e1f59c04ad190382259705b1b54 (DEFAULT_TABLE table=default.Table_0913077ace074f11bf8a6222338489c6 [id=e048189f6ef441ce8e17c60f0f5b0cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.290135 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4497e1f59c04ad190382259705b1b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.289921 5334 tablet_service.cc:1467] Processing CreateTablet for tablet 8d17615af06b4144a0d7d74c64416451 (DEFAULT_TABLE table=default.Table_f10f80e85d6c4f87b43c1a53b888716c [id=20c27e5e63df49a8bcf6faef31940880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.290851 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d17615af06b4144a0d7d74c64416451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.291510 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 0d8357611ced42db91bd1e416f0e26b3 (DEFAULT_TABLE table=default.Table_a446b3e7bb7944b0a5b119dd6a1b6d7c [id=4bfd4a2482364643a6d15f7a638f2eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.292425 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d8357611ced42db91bd1e416f0e26b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.295221 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 3024717477814906a3d861df35e4cb6c (DEFAULT_TABLE table=default.Table_e4788552c1a8457a8d34951f6827c5e4 [id=a14f97b5af3741bebd7d88f5602675f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.296118 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3024717477814906a3d861df35e4cb6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.299196 7199 ts_tablet_manager.cc:1918] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:38.299649 7199 log.cc:1196] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/9423b380a62c467895e7fdff14756586
I20260808 13:35:38.299521 5354 tablet_service.cc:1467] Processing CreateTablet for tablet ef2b958e376140f7949aa04d49a4aec6 (DEFAULT_TABLE table=default.Table_82d49330ca914820a6cb8b3591280a90 [id=dc1f8a5aa37a4ad99be0715831a680fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.300392 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef2b958e376140f7949aa04d49a4aec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.300483 7199 ts_tablet_manager.cc:1939] T 9423b380a62c467895e7fdff14756586 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:38.306356 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:38.310940 7881 raft_consensus.cc:482] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5cd987c3dbdb4168beb73cd7e577c7cb)
I20260808 13:35:38.311510 7881 raft_consensus.cc:504] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.316684 7881 leader_election.cc:288] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:38.317473 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:38.324152 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_41de7b9190a34c0f97e4c0d7fcc14a49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:38.325805 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41de7b9190a34c0f97e4c0d7fcc14a49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:38.327221 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet b1847d0fecb843cdbc833ad674a503a4 (table default.Table_2d75fe6a74ad43789e515c67b5167ab3 [id=29db8794594e4221be8974258b5127e0]). Expected version 1 got 0
I20260808 13:35:38.331292 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:38.331955 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:38.332662 4842 catalog_manager.cc:5495] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:38.150456 7013 rpcz_store.cc:276] 0808 13:35:34.853310 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.853491 (+ 181us) service_pool.cc:227] Handling call
0808 13:35:38.149854 (+3296363us) ts_tablet_manager.cc:1145] Deleting tablet 6e70a5cf88d94b36913a6d1ccfcb979a
0808 13:35:38.149893 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.150117 (+ 224us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3271830}
I20260808 13:35:38.339747 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013]). Expected version 6 got 0
I20260808 13:35:38.340551 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:38.340988 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 328d361c336848689c6906ca6fe2ccc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:35 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:38.341596 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet d8b07b7d31ef41e484389c0f007598f0 (table default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]). Expected version 1 got 0
I20260808 13:35:38.342460 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:38.342680 5337 tablet_service.cc:1467] Processing CreateTablet for tablet 64be4bede9d04d2db0940a75e0e9b2fb (DEFAULT_TABLE table=default.Table_6b9bc590a69746909348fbcbb7856cdf [id=03f2c8cadea14860aee2cd51574379b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.344511 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 328d361c336848689c6906ca6fe2ccc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:35 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:38.344631 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3ad9ca75b541798bc04d7701128faa (DEFAULT_TABLE table=default.Table_642fc6953dc8471cbd814ae6fda0cc6a [id=9136fce84d3d447a999d24131713ce4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.344043 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:38.347631 6046 tablet_bootstrap.cc:654] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:38.382021 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 1ba5fa465ec04bbb801a821a71a8f87d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ba5fa465ec04bbb801a821a71a8f87d already in progress: creating tablet
I20260808 13:35:38.348326 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 364) took 2701 ms. Trace:
I20260808 13:35:38.351652 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3ad9ca75b541798bc04d7701128faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.387462 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64be4bede9d04d2db0940a75e0e9b2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.372586 5338 tablet_service.cc:1467] Processing CreateTablet for tablet 14a9373000e24e75858e6d7457587264 (DEFAULT_TABLE table=default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.391571 5338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a9373000e24e75858e6d7457587264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.374442 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 7b128feff212456faa1769efd96f6bd8 (DEFAULT_TABLE table=default.Table_034b94572e274eff86802d9cae6c907d [id=087eec2212324876bebfc252a552d074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.395651 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b128feff212456faa1769efd96f6bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.376304 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66b38e83dab4bf1a3d2d1e088a96988" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:38.399147 5378 raft_consensus.cc:3037] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:38.378043 5353 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8cf06462484c99a860762a82f7ad64 with delete_type TABLET_DATA_DELETED (replaced by tablet 16736500f94c43d3a3163f61b7606aad) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:38.411428 5325 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:46324 (request call id 36) took 1126 ms. Trace:
I20260808 13:35:38.411904 5325 rpcz_store.cc:276] 0808 13:35:37.284839 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:37.285702 (+ 863us) service_pool.cc:227] Handling call
0808 13:35:38.411354 (+1125652us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:38.424602 5378 raft_consensus.cc:2454] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:38.425398 7013 rpcz_store.cc:276] 0808 13:35:35.646994 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.647099 (+ 105us) service_pool.cc:227] Handling call
0808 13:35:38.340117 (+2693018us) ts_tablet_manager.cc:1145] Deleting tablet d3682c9c656c4556bc13ae3aeddd7c5e
0808 13:35:38.347900 (+ 7783us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.348270 (+ 370us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2671094}
I20260808 13:35:38.426524 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 365) took 2766 ms. Trace:
I20260808 13:35:38.426862 7013 rpcz_store.cc:276] 0808 13:35:35.659956 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.662907 (+ 2951us) service_pool.cc:227] Handling call
0808 13:35:38.426253 (+2763346us) ts_tablet_manager.cc:1145] Deleting tablet 31fc9fdb5ef543da833f9af6ca5807a9
0808 13:35:38.426297 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.426476 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2756888}
I20260808 13:35:38.427954 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 363) took 2781 ms. Trace:
I20260808 13:35:38.428217 7013 rpcz_store.cc:276] 0808 13:35:35.646665 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.646800 (+ 135us) service_pool.cc:227] Handling call
0808 13:35:38.427711 (+2780911us) ts_tablet_manager.cc:1145] Deleting tablet ddfd9e3f002445738df24ebb0a333440
0808 13:35:38.427752 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.427908 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2752914}
I20260808 13:35:38.429175 7013 tablet_replica.cc:330] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:38.430030 7013 raft_consensus.cc:2229] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:38.430089 5598 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3ad9ca75b541798bc04d7701128faa (DEFAULT_TABLE table=default.Table_642fc6953dc8471cbd814ae6fda0cc6a [id=9136fce84d3d447a999d24131713ce4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.433638 7844 raft_consensus.cc:482] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:38.434196 7844 raft_consensus.cc:504] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.438864 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9bc338093f403881d329a7e913c9f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:20 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:38.439754 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:38.440433 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 9423b380a62c467895e7fdff14756586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:38.440713 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 14a9373000e24e75858e6d7457587264 (DEFAULT_TABLE table=default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.442085 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 7b128feff212456faa1769efd96f6bd8 (DEFAULT_TABLE table=default.Table_034b94572e274eff86802d9cae6c907d [id=087eec2212324876bebfc252a552d074]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:38.447461 5285 leader_election.cc:341] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 9423b380a62c467895e7fdff14756586
I20260808 13:35:38.460721 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 340) took 3490 ms. Trace:
I20260808 13:35:38.461095 7199 rpcz_store.cc:276] 0808 13:35:34.970319 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:34.972430 (+ 2111us) service_pool.cc:227] Handling call
0808 13:35:38.080223 (+3107793us) ts_tablet_manager.cc:1145] Deleting tablet 9423b380a62c467895e7fdff14756586
0808 13:35:38.080275 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.219822 (+139547us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.298925 (+ 79103us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.460637 (+161712us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":622,"tablet-delete.queue_time_us":3107044}
I20260808 13:35:38.463819 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 9423b380a62c467895e7fdff14756586 (table default.Table_1fe1ec319d834d319cd9c35e801b1c91 [id=0f9219b97bfc4351ad5b584cfdec058f]) successfully deleted
W20260808 13:35:38.467038 5283 leader_election.cc:341] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: 9423b380a62c467895e7fdff14756586
I20260808 13:35:38.467561 5283 leader_election.cc:302] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:38.468356 7882 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:35:38.462566 7199 tablet_replica.cc:330] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:38.480940 7199 raft_consensus.cc:2229] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:38.481523 7199 raft_consensus.cc:2258] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:38.430527 7013 raft_consensus.cc:2258] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:38.489015 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:38.489737 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013]). Expected version 6 got 0
I20260808 13:35:38.491176 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:38.494278 7716 raft_consensus.cc:2726] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:38.496811 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 8e843a211ab24792b9a475cf08e8c49b (DEFAULT_TABLE table=default.Table_8a69b0e5577e4c6f9c76504751d2c1a4 [id=1b8762a3419f47bb9c060294f70d62aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.497257 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet b1847d0fecb843cdbc833ad674a503a4 (table default.Table_2d75fe6a74ad43789e515c67b5167ab3 [id=29db8794594e4221be8974258b5127e0]). Expected version 1 got 0
I20260808 13:35:38.498128 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e843a211ab24792b9a475cf08e8c49b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:38.499290 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 51243.5us
W20260808 13:35:38.497637 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 3451d72548a140c68a6ccc2b566a09b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3451d72548a140c68a6ccc2b566a09b1 already in progress: creating tablet
W20260808 13:35:38.501068 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 61dfd47db0a74ed8b7228b559dd68459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 61dfd47db0a74ed8b7228b559dd68459 already in progress: creating tablet
W20260808 13:35:38.501652 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet cb7f6487e28f4aaa94d448be71f8a3c2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb7f6487e28f4aaa94d448be71f8a3c2 already in progress: creating tablet
W20260808 13:35:38.502197 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 293e9540498e420d8ec873472d0d29fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 293e9540498e420d8ec873472d0d29fc already in progress: creating tablet
W20260808 13:35:38.502799 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 1f0a6d9bd2c74d0790badd7977f027e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f0a6d9bd2c74d0790badd7977f027e3 already in progress: creating tablet
W20260808 13:35:38.503576 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 6e70a5cf88d94b36913a6d1ccfcb979a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e70a5cf88d94b36913a6d1ccfcb979a already in progress: creating tablet
I20260808 13:35:38.505172 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:38.510468 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:38.491665 4840 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fc33bfcdedd744039ea0ff6c698d01cc" } new_table_name: "default.Table_62ad61e62a5b49b480ec08bc990efa85" modify_external_catalogs: true
I20260808 13:35:38.522683 5771 leader_election.cc:302] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:35:38.524127 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet d3682c9c656c4556bc13ae3aeddd7c5e because the tablet was not found. No further retry: Not found: Tablet not found: d3682c9c656c4556bc13ae3aeddd7c5e
W20260808 13:35:38.525488 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 31fc9fdb5ef543da833f9af6ca5807a9 because the tablet was not found. No further retry: Not found: Tablet not found: 31fc9fdb5ef543da833f9af6ca5807a9
W20260808 13:35:38.526840 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet ddfd9e3f002445738df24ebb0a333440 because the tablet was not found. No further retry: Not found: Tablet not found: ddfd9e3f002445738df24ebb0a333440
I20260808 13:35:38.539830 6033 tablet_bootstrap.cc:654] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:38.539928 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a9373000e24e75858e6d7457587264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.543045 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b128feff212456faa1769efd96f6bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.540825 7560 raft_consensus.cc:2781] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:38.547569 7560 raft_consensus.cc:686] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:38.548137 7911 tablet.cc:1722] T 4b7284afe0294644a368cf51451c7ab6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260808 13:35:38.548403 7560 consensus_queue.cc:237] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:38.552335 5829 consensus_meta.cc:287] T 8e843a211ab24792b9a475cf08e8c49b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.508s user 0.007s sys 0.000s
I20260808 13:35:38.553992 5610 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8cf06462484c99a860762a82f7ad64 with delete_type TABLET_DATA_DELETED (replaced by tablet 16736500f94c43d3a3163f61b7606aad) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:38.554569 5616 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:38.555429 7199 ts_tablet_manager.cc:1905] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:38.556020 7896 raft_consensus.cc:482] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:38.556423 7896 raft_consensus.cc:504] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.561713 7704 raft_consensus.cc:482] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:38.558409 7896 leader_election.cc:288] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:38.563158 5598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3ad9ca75b541798bc04d7701128faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.562151 7704 raft_consensus.cc:504] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.567379 7704 leader_election.cc:288] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:38.568293 7882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Bootstrap starting.
I20260808 13:35:38.472378 7844 leader_election.cc:288] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:38.579119 5595 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.743s user 0.007s sys 0.016s
W20260808 13:35:38.579582 5595 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.788s user 0.007s sys 0.016s
W20260808 13:35:38.582053 6026 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.687s user 0.013s sys 0.017s
W20260808 13:35:38.582521 6026 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.718s user 0.013s sys 0.018s
I20260808 13:35:38.584965 6026 tablet_bootstrap.cc:492] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:38.586294 6026 ts_tablet_manager.cc:1397] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.831s user 0.021s sys 0.022s
I20260808 13:35:38.590041 7013 ts_tablet_manager.cc:1905] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:38.590479 6046 tablet_bootstrap.cc:492] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:38.590893 6046 ts_tablet_manager.cc:1397] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.429s user 0.011s sys 0.006s
I20260808 13:35:38.595278 6046 raft_consensus.cc:348] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.598315 6046 raft_consensus.cc:374] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:38.598676 6046 raft_consensus.cc:729] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:38.606510 6046 consensus_queue.cc:260] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.613533 6026 raft_consensus.cc:348] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.614177 6026 raft_consensus.cc:374] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:38.616431 7906 tablet_replica.cc:330] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:38.618103 7906 raft_consensus.cc:2229] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:38.622370 7013 ts_tablet_manager.cc:1918] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:38.622767 7013 log.cc:1196] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/2cd6945ecccc4684a3fa5b2a1f07057f
I20260808 13:35:38.623494 7013 ts_tablet_manager.cc:1939] T 2cd6945ecccc4684a3fa5b2a1f07057f P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:38.608889 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72c4ddec62c242c88b3a759d8361cd30" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:38.630537 6046 ts_tablet_manager.cc:1428] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.039s user 0.005s sys 0.000s
I20260808 13:35:38.638536 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 367) took 2902 ms. Trace:
I20260808 13:35:38.638827 6046 tablet_bootstrap.cc:492] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:38.638873 7013 rpcz_store.cc:276] 0808 13:35:35.736128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.738009 (+ 1881us) service_pool.cc:227] Handling call
0808 13:35:38.429011 (+2691002us) ts_tablet_manager.cc:1145] Deleting tablet 2cd6945ecccc4684a3fa5b2a1f07057f
0808 13:35:38.429057 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.614082 (+185025us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.622148 (+ 8066us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.638476 (+ 16328us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8408,"tablet-delete.queue_time_us":2690444}
I20260808 13:35:38.639955 7013 tablet_replica.cc:330] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:38.640640 7013 raft_consensus.cc:2229] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:38.641113 7013 raft_consensus.cc:2258] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:38.643067 7013 ts_tablet_manager.cc:1905] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:38.644711 6046 tablet_bootstrap.cc:654] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:38.648269 7882 log.cc:824] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Log is configured to *not* fsync() on all Append() calls
W20260808 13:35:38.664044 5771 leader_election.cc:341] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:38.667191 6046 tablet_bootstrap.cc:492] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:38.667757 6046 ts_tablet_manager.cc:1397] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.029s user 0.010s sys 0.000s
I20260808 13:35:38.668510 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 2cd6945ecccc4684a3fa5b2a1f07057f (table default.Table_b75082dac94348fab668160895573637 [id=56f4695998f34301b8528a79b16e2dd2]) successfully deleted
I20260808 13:35:38.670325 5378 raft_consensus.cc:3037] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:38.670450 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b57966a9b7b4673ab9f98720cfd456d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:38.674687 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72c4ddec62c242c88b3a759d8361cd30" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:38.678233 7906 raft_consensus.cc:2258] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:38.678490 6046 raft_consensus.cc:348] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.658455 7199 ts_tablet_manager.cc:1918] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:38.679407 7199 log.cc:1196] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/e00f2e0b98af431d99894e8778257bc4
I20260808 13:35:38.680222 7199 ts_tablet_manager.cc:1939] T e00f2e0b98af431d99894e8778257bc4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:38.680454 6046 raft_consensus.cc:374] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:38.680738 7906 ts_tablet_manager.cc:1905] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:38.680773 6046 raft_consensus.cc:729] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:38.681043 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 353) took 3131 ms. Trace:
I20260808 13:35:38.681442 6046 consensus_queue.cc:260] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.681504 7199 rpcz_store.cc:276] 0808 13:35:35.549355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.549511 (+ 156us) service_pool.cc:227] Handling call
0808 13:35:38.462414 (+2912903us) ts_tablet_manager.cc:1145] Deleting tablet e00f2e0b98af431d99894e8778257bc4
0808 13:35:38.462451 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.610332 (+147881us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.658238 (+ 47906us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.680989 (+ 22751us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2912231}
W20260808 13:35:38.602479 5551 leader_election.cc:341] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:38.683043 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 80737.2us
W20260808 13:35:38.684692 5551 consensus_peers.cc:487] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:38.688457 6046 ts_tablet_manager.cc:1428] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260808 13:35:38.689404 6046 tablet_bootstrap.cc:492] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:38.614440 6026 raft_consensus.cc:729] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
W20260808 13:35:38.677062 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 260479us
I20260808 13:35:38.694576 6046 tablet_bootstrap.cc:654] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:38.703110 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet e00f2e0b98af431d99894e8778257bc4 (table default.Table_8523d4a015194c64b22f66a22b0dcc06 [id=b285a956530e4fb4a5d1632a45d87c0e]) successfully deleted
I20260808 13:35:38.731487 7013 ts_tablet_manager.cc:1918] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:38.732016 7013 log.cc:1196] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/127e8d0323954ef788018bcb91582951
I20260808 13:35:38.740303 7013 ts_tablet_manager.cc:1939] T 127e8d0323954ef788018bcb91582951 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:38.742370 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 370) took 3005 ms. Trace:
I20260808 13:35:38.742764 7013 rpcz_store.cc:276] 0808 13:35:35.737013 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.739662 (+ 2649us) service_pool.cc:227] Handling call
0808 13:35:38.639830 (+2900168us) ts_tablet_manager.cc:1145] Deleting tablet 127e8d0323954ef788018bcb91582951
0808 13:35:38.639866 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.661082 (+ 21216us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.731256 (+ 70174us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.742314 (+ 11058us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2899695}
I20260808 13:35:38.744179 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 371) took 3006 ms. Trace:
I20260808 13:35:38.744534 7013 rpcz_store.cc:276] 0808 13:35:35.737261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.740214 (+ 2953us) service_pool.cc:227] Handling call
0808 13:35:38.743927 (+3003713us) ts_tablet_manager.cc:1145] Deleting tablet 6826e4e4db2043b3a4fe9e7a9359cab6
0808 13:35:38.743965 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.744134 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3003267}
I20260808 13:35:38.745597 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 372) took 3008 ms. Trace:
I20260808 13:35:38.745932 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 127e8d0323954ef788018bcb91582951 (table default.Table_23396b27c08444c7981e55a66139f6b6 [id=2004b79d44ab440ba8d0004db91b6bce]) successfully deleted
I20260808 13:35:38.745940 7013 rpcz_store.cc:276] 0808 13:35:35.737557 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.740731 (+ 3174us) service_pool.cc:227] Handling call
0808 13:35:38.745371 (+3004640us) ts_tablet_manager.cc:1145] Deleting tablet a8e18e51c2e24d3caa3d29e6645e46ae
0808 13:35:38.745406 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.745556 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3004191}
W20260808 13:35:38.746892 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 6826e4e4db2043b3a4fe9e7a9359cab6 because the tablet was not found. No further retry: Not found: Tablet not found: 6826e4e4db2043b3a4fe9e7a9359cab6
I20260808 13:35:38.747079 7013 tablet_replica.cc:330] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
W20260808 13:35:38.747946 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet a8e18e51c2e24d3caa3d29e6645e46ae because the tablet was not found. No further retry: Not found: Tablet not found: a8e18e51c2e24d3caa3d29e6645e46ae
I20260808 13:35:38.748589 7013 raft_consensus.cc:2229] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:38.749151 7013 raft_consensus.cc:2258] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:38.753669 7199 tablet_replica.cc:330] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:38.754632 7199 raft_consensus.cc:2229] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:38.755501 7199 raft_consensus.cc:2258] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:38.757285 7199 ts_tablet_manager.cc:1905] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:38.764521 7013 ts_tablet_manager.cc:1905] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:38.775600 7882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=5 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:35:38.776937 7882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Bootstrap complete.
I20260808 13:35:38.754940 6026 consensus_queue.cc:260] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.783601 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b57966a9b7b4673ab9f98720cfd456d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:38.787072 7918 tablet.cc:1722] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:38.794639 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72c4ddec62c242c88b3a759d8361cd30" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:38.795321 5877 raft_consensus.cc:2454] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:38.809667 7013 ts_tablet_manager.cc:1918] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:38.810141 7013 log.cc:1196] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/69642fdba1f145438b04eae9f8154101
I20260808 13:35:38.810043 7911 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:50600 (request call id 355) took 1096 ms. Trace:
I20260808 13:35:38.810937 7013 ts_tablet_manager.cc:1939] T 69642fdba1f145438b04eae9f8154101 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:38.754187 7902 consensus_queue.cc:1035] T 16a6552c871b4b81b6520cb084d359a2 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.106s
I20260808 13:35:38.814783 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 368) took 3078 ms. Trace:
W20260808 13:35:38.814862 5551 leader_election.cc:341] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:38.815256 5551 leader_election.cc:302] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:38.816332 5551 leader_election.cc:341] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:38.817104 5551 leader_election.cc:341] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:38.817366 7893 raft_consensus.cc:2726] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:38.817420 5551 leader_election.cc:302] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:38.818747 6026 ts_tablet_manager.cc:1428] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.207s user 0.006s sys 0.000s
W20260808 13:35:38.819132 5551 leader_election.cc:341] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:38.819600 5551 leader_election.cc:302] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:38.819751 6026 tablet_bootstrap.cc:492] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:38.820477 7891 raft_consensus.cc:2726] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:38.820855 7013 rpcz_store.cc:276] 0808 13:35:35.736426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.741297 (+ 4871us) service_pool.cc:227] Handling call
0808 13:35:38.746947 (+3005650us) ts_tablet_manager.cc:1145] Deleting tablet 69642fdba1f145438b04eae9f8154101
0808 13:35:38.746982 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.786525 (+ 39543us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.809471 (+ 22946us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.814719 (+ 5248us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3005170}
I20260808 13:35:38.825131 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 69642fdba1f145438b04eae9f8154101 (table default.Table_41f3478fe8c94fbfaf7fcbdee22ce987 [id=e3e121c36ff44ea0a3a98185c8cce381]) successfully deleted
I20260808 13:35:38.825906 7830 raft_consensus.cc:482] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:38.826397 7013 tablet_replica.cc:330] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:38.826318 7830 raft_consensus.cc:504] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.827152 7013 raft_consensus.cc:2229] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:38.827585 7013 raft_consensus.cc:2258] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:38.810388 7911 rpcz_store.cc:276] 0808 13:35:37.713521 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:37.713658 (+ 137us) service_pool.cc:227] Handling call
0808 13:35:38.809964 (+1096306us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:37.714921 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:37.715315 (+ 394us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:37.715351 (+ 36us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:37.715402 (+ 51us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:37.715540 (+ 138us) alter_schema_op.cc:126] START. Timestamp: 1786196137715417
0808 13:35:37.715563 (+ 23us) op_driver.cc:331] REPLICATION: starting
0808 13:35:37.716133 (+ 570us) log.cc:842] Serialized 173 byte log entry
0808 13:35:38.432709 (+716576us) op_driver.cc:444] REPLICATION: finished
0808 13:35:38.547886 (+115177us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:38.549458 (+ 1572us) log.cc:842] Serialized 5 byte log entry
0808 13:35:38.808062 (+258604us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.808975 (+ 913us) log.cc:842] Serialized 14 byte log entry
0808 13:35:38.809806 (+ 831us) alter_schema_op.cc:71] Released schema lock
0808 13:35:38.809841 (+ 35us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":114914,"prepare.queue_time_us":453,"prepare.run_cpu_time_us":8615,"prepare.run_wall_time_us":81977,"raft.queue_time_us":146,"raft.run_cpu_time_us":1203,"raft.run_wall_time_us":1203,"replication_time_us":716973,"thread_start_us":156304,"threads_started":2,"wal-append.queue_time_us":73414}]]}
I20260808 13:35:38.828263 7830 leader_election.cc:288] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:38.829492 7013 ts_tablet_manager.cc:1905] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:38.831826 5551 leader_election.cc:341] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:38.832178 5551 leader_election.cc:302] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:38.833086 5551 leader_election.cc:341] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:38.833981 5551 leader_election.cc:341] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:38.834803 7893 raft_consensus.cc:2726] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:38.835301 7893 raft_consensus.cc:2726] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:38.836862 7896 consensus_queue.cc:1035] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:38.840003 5550 leader_election.cc:341] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:38.840345 5550 leader_election.cc:302] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:38.841495 5550 leader_election.cc:302] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:38.843776 7891 raft_consensus.cc:2726] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:38.850471 7893 raft_consensus.cc:2781] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:38.850824 7893 raft_consensus.cc:482] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:38.851101 7893 raft_consensus.cc:3037] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:38.855209 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d8d5a3ca724141b58e4e2cea23be68" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:38.857614 5771 consensus_peers.cc:487] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 69642fdba1f145438b04eae9f8154101. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:38.859794 7937 tablet.cc:1722] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:38.870793 7896 consensus_queue.cc:1035] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:38.829762 6046 tablet_bootstrap.cc:492] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:38.884793 6046 ts_tablet_manager.cc:1397] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.196s user 0.020s sys 0.003s
I20260808 13:35:38.885183 5354 tablet_service.cc:1467] Processing CreateTablet for tablet b704d251093c40eba4cc02140046256e (DEFAULT_TABLE table=default.Table_03cf1a01ebc14024a283b7e5ed420074 [id=a45370bd3bbc48279db129789b269480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.887045 5603 tablet_service.cc:1467] Processing CreateTablet for tablet b704d251093c40eba4cc02140046256e (DEFAULT_TABLE table=default.Table_03cf1a01ebc14024a283b7e5ed420074 [id=a45370bd3bbc48279db129789b269480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.887081 6046 raft_consensus.cc:348] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.887741 6046 raft_consensus.cc:374] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:38.887991 6046 raft_consensus.cc:729] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:38.888370 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b704d251093c40eba4cc02140046256e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:38.830930 5285 leader_election.cc:341] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 56d8d5a3ca724141b58e4e2cea23be68
W20260808 13:35:38.888681 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 57781.4us
I20260808 13:35:38.890494 5283 leader_election.cc:302] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:38.891500 7914 raft_consensus.cc:2726] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:38.891922 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b704d251093c40eba4cc02140046256e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.892441 6046 consensus_queue.cc:260] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:38.897871 5829 tablet_service.cc:1467] Processing CreateTablet for tablet b704d251093c40eba4cc02140046256e (DEFAULT_TABLE table=default.Table_03cf1a01ebc14024a283b7e5ed420074 [id=a45370bd3bbc48279db129789b269480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.898968 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b704d251093c40eba4cc02140046256e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.899412 6026 tablet_bootstrap.cc:654] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:38.877792 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:38.903421 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:38.903292 5332 tablet_service.cc:1467] Processing CreateTablet for tablet df27489ae6e2456996671e8fa57af566 (DEFAULT_TABLE table=default.Table_702490ba7901458b954c15c4637424b0 [id=1e6b56dff3614e5fb96523262224c209]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.904379 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df27489ae6e2456996671e8fa57af566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.906068 5614 tablet_service.cc:1467] Processing CreateTablet for tablet df27489ae6e2456996671e8fa57af566 (DEFAULT_TABLE table=default.Table_702490ba7901458b954c15c4637424b0 [id=1e6b56dff3614e5fb96523262224c209]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.907205 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df27489ae6e2456996671e8fa57af566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.912863 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72c4ddec62c242c88b3a759d8361cd30" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:38.924227 6046 ts_tablet_manager.cc:1428] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.039s user 0.005s sys 0.001s
I20260808 13:35:38.934974 6046 tablet_bootstrap.cc:492] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:38.753044 6033 tablet_bootstrap.cc:492] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:38.936084 6033 ts_tablet_manager.cc:1397] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.705s user 0.017s sys 0.009s
I20260808 13:35:38.938761 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:38.939215 5378 raft_consensus.cc:3037] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:38.939793 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013]). Expected version 6 got 0
I20260808 13:35:38.940542 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:38.941208 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:38.941884 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:38.946270 5338 tablet_service.cc:1467] Processing CreateTablet for tablet 76f6574b348a44e193245f89b67ae061 (DEFAULT_TABLE table=default.Table_4579204171cf47c6b668cedc2efaa649 [id=71918a8fb02640a48c7838e607490743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.951653 5340 tablet_service.cc:1467] Processing CreateTablet for tablet bf780c8e7ec949bd805e7fc9df5f321e (DEFAULT_TABLE table=default.Table_c06c463992cf45db8dacc3018d8bcb8a [id=00bfe57fc39c432690f17b74eab8d1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.952875 5842 tablet_service.cc:1467] Processing CreateTablet for tablet df27489ae6e2456996671e8fa57af566 (DEFAULT_TABLE table=default.Table_702490ba7901458b954c15c4637424b0 [id=1e6b56dff3614e5fb96523262224c209]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.952744 5826 tablet_service.cc:1467] Processing CreateTablet for tablet 76f6574b348a44e193245f89b67ae061 (DEFAULT_TABLE table=default.Table_4579204171cf47c6b668cedc2efaa649 [id=71918a8fb02640a48c7838e607490743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.953913 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df27489ae6e2456996671e8fa57af566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.955842 5826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f6574b348a44e193245f89b67ae061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.960512 7937 tablet.cc:1722] T 4b7284afe0294644a368cf51451c7ab6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260808 13:35:38.969198 5843 tablet_service.cc:1467] Processing CreateTablet for tablet bf780c8e7ec949bd805e7fc9df5f321e (DEFAULT_TABLE table=default.Table_c06c463992cf45db8dacc3018d8bcb8a [id=00bfe57fc39c432690f17b74eab8d1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.969437 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet b1847d0fecb843cdbc833ad674a503a4 (table default.Table_2d75fe6a74ad43789e515c67b5167ab3 [id=29db8794594e4221be8974258b5127e0]). Expected version 1 got 0
I20260808 13:35:38.970585 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf780c8e7ec949bd805e7fc9df5f321e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.972294 7199 ts_tablet_manager.cc:1918] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:38.972545 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 76f6574b348a44e193245f89b67ae061 (DEFAULT_TABLE table=default.Table_4579204171cf47c6b668cedc2efaa649 [id=71918a8fb02640a48c7838e607490743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.978762 4832 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea] alter complete (version 1)
I20260808 13:35:38.978895 7199 log.cc:1196] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/fc9b2e69fe27431dac9eb5ecb9e3e23e
I20260808 13:35:38.979655 7199 ts_tablet_manager.cc:1939] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:38.982405 5613 tablet_service.cc:1467] Processing CreateTablet for tablet bf780c8e7ec949bd805e7fc9df5f321e (DEFAULT_TABLE table=default.Table_c06c463992cf45db8dacc3018d8bcb8a [id=00bfe57fc39c432690f17b74eab8d1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:38.984797 4835 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532] alter complete (version 1)
I20260808 13:35:38.986478 5338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f6574b348a44e193245f89b67ae061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:38.991852 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf780c8e7ec949bd805e7fc9df5f321e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:38.995431 5772 consensus_peers.cc:487] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56d8d5a3ca724141b58e4e2cea23be68. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:38.917330 7893 raft_consensus.cc:504] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.003554 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 354) took 3449 ms. Trace:
I20260808 13:35:39.004387 5598 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d8d5a3ca724141b58e4e2cea23be68 with delete_type TABLET_DATA_DELETED (replaced by tablet 14a9373000e24e75858e6d7457587264) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:39.005112 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:39.010937 7896 raft_consensus.cc:482] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:39.009219 7199 rpcz_store.cc:276] 0808 13:35:35.554316 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.554455 (+ 139us) service_pool.cc:227] Handling call
0808 13:35:38.682540 (+3128085us) ts_tablet_manager.cc:1145] Deleting tablet fc9b2e69fe27431dac9eb5ecb9e3e23e
0808 13:35:38.682580 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.839265 (+156685us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:38.943392 (+104127us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.003495 (+ 60103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3127493}
I20260808 13:35:39.011404 7896 raft_consensus.cc:504] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:38.994550 4833 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 49) took 2408 ms (client timeout 1000 ms). Trace:
I20260808 13:35:39.012326 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet fc9b2e69fe27431dac9eb5ecb9e3e23e (table default.Table_13f833fb48504c76865676e23461e6e5 [id=0a8dbdef483f48b68019ea98b335a11e]) successfully deleted
I20260808 13:35:39.013433 7896 leader_election.cc:288] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:39.013702 7199 tablet_replica.cc:330] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:39.015203 7893 leader_election.cc:288] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:39.015982 5635 raft_consensus.cc:2379] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
W20260808 13:35:39.018286 4842 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet d8b07b7d31ef41e484389c0f007598f0 (table default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet d8b07b7d31ef41e484389c0f007598f0 consensus state has no leader
I20260808 13:35:39.018728 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet 369a942248bd4cec9ee9170fe55601a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:39.019767 7199 raft_consensus.cc:2229] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:39.020391 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f6574b348a44e193245f89b67ae061. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:39.021082 4842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 50) took 1167 ms (client timeout 1000 ms). Trace:
W20260808 13:35:39.021400 4842 rpcz_store.cc:269] 0808 13:35:37.853599 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:37.915489 (+ 61890us) service_pool.cc:227] Handling call
0808 13:35:37.943908 (+ 28419us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:38.414279 (+470371us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.414305 (+ 26us) write_op.cc:454] Released schema lock
0808 13:35:39.021018 (+606713us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:38.352158 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:38.352525 (+ 367us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:38.352549 (+ 24us) write_op.cc:435] Acquired schema lock
0808 13:35:38.352562 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:38.352822 (+ 260us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:38.352861 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:38.352882 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:38.353010 (+ 128us) tablet.cc:699] Row locks acquired
0808 13:35:38.353022 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:38.353127 (+ 105us) write_op.cc:270] Start()
0808 13:35:38.353201 (+ 74us) write_op.cc:276] Timestamp: P: 1786196138353054 usec, L: 0
0808 13:35:38.353216 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:38.353678 (+ 462us) log.cc:842] Serialized 527 byte log entry
0808 13:35:38.373151 (+ 19473us) op_driver.cc:444] REPLICATION: finished
0808 13:35:38.396445 (+ 23294us) write_op.cc:301] APPLY: starting
0808 13:35:38.396501 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:38.396587 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:38.396686 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:38.396881 (+ 195us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:38.399348 (+ 2467us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:38.399392 (+ 44us) write_op.cc:312] APPLY: finished
0808 13:35:38.399551 (+ 159us) log.cc:842] Serialized 23 byte log entry
0808 13:35:38.400618 (+ 1067us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.400698 (+ 80us) write_op.cc:454] Released schema lock
0808 13:35:38.400751 (+ 53us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":5817,"reported_tablets":26,"spinlock_wait_cycles":41376512,"child_traces":[["op",{"apply.queue_time_us":22482,"apply.run_cpu_time_us":1399,"apply.run_wall_time_us":5157,"num_ops":1,"prepare.queue_time_us":4426,"prepare.run_cpu_time_us":1780,"prepare.run_wall_time_us":1781,"raft.queue_time_us":164,"raft.run_cpu_time_us":682,"raft.run_wall_time_us":1564,"replication_time_us":19859}]]}
I20260808 13:35:39.026800 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf780c8e7ec949bd805e7fc9df5f321e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:39.012205 4833 rpcz_store.cc:269] 0808 13:35:36.585652 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.661410 (+ 75758us) service_pool.cc:227] Handling call
0808 13:35:38.994474 (+2333064us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:37.528265 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:37.528740 (+ 475us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:37.528780 (+ 40us) write_op.cc:435] Acquired schema lock
0808 13:35:37.528793 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:37.529161 (+ 368us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:37.529206 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:37.529246 (+ 40us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:37.529615 (+ 369us) tablet.cc:699] Row locks acquired
0808 13:35:37.529634 (+ 19us) write_op.cc:260] PREPARE: finished
0808 13:35:37.529775 (+ 141us) write_op.cc:270] Start()
0808 13:35:37.529865 (+ 90us) write_op.cc:276] Timestamp: P: 1786196137529717 usec, L: 0
0808 13:35:37.529885 (+ 20us) op_driver.cc:331] REPLICATION: starting
0808 13:35:37.530386 (+ 501us) log.cc:842] Serialized 901 byte log entry
0808 13:35:37.540572 (+ 10186us) op_driver.cc:444] REPLICATION: finished
0808 13:35:37.541025 (+ 453us) write_op.cc:301] APPLY: starting
0808 13:35:37.541106 (+ 81us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:37.541277 (+ 171us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:37.541378 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:37.541753 (+ 375us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:37.542009 (+ 256us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:37.542096 (+ 87us) write_op.cc:312] APPLY: finished
0808 13:35:37.542346 (+ 250us) log.cc:842] Serialized 29 byte log entry
0808 13:35:37.542561 (+ 215us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.542733 (+ 172us) write_op.cc:454] Released schema lock
0808 13:35:37.542794 (+ 61us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:37.543203 (+ 409us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.543243 (+ 40us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":267,"apply.run_cpu_time_us":2449,"apply.run_wall_time_us":2448,"num_ops":2,"prepare.queue_time_us":120,"prepare.run_cpu_time_us":2457,"prepare.run_wall_time_us":2459,"raft.queue_time_us":261,"raft.run_cpu_time_us":1444,"raft.run_wall_time_us":1444,"replication_time_us":10583}]]}
I20260808 13:35:39.028899 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_e70822ac71fa4d0e9205367df025ad3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:39.033393 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e70822ac71fa4d0e9205367df025ad3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:39.039776 5720 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:35:39.040441 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 11cd8f12cda94c4eb9a848f002a4020b
I20260808 13:35:39.041909 4839 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 416) took 2022 ms. Trace:
I20260808 13:35:39.045039 7918 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:50600 (request call id 356) took 1256 ms. Trace:
I20260808 13:35:39.046701 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cd8f12cda94c4eb9a848f002a4020b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:37 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:39.049764 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cd8f12cda94c4eb9a848f002a4020b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:37 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:39.042251 4839 rpcz_store.cc:276] 0808 13:35:37.019646 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:37.414044 (+394398us) service_pool.cc:227] Handling call
0808 13:35:37.423595 (+ 9551us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff466 00007f227bd40b34 00007f227bd40ac2 00007f227bd409f1 00007f227bd4096c 00007f227bd3fc22 00007f2279fde94c 00007f2279fddd86
0808 13:35:37.424972 (+ 1377us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:37.425867 (+ 895us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:37.426044 (+ 177us) catalog_manager.cc:2761] Locking tablets
0808 13:35:38.236218 (+810174us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:38.277313 (+ 41095us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:38.277561 (+ 248us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:39.040348 (+762787us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:39.041851 (+ 1503us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:38.242918 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:38.243296 (+ 378us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:38.243323 (+ 27us) write_op.cc:435] Acquired schema lock
0808 13:35:38.243336 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:38.243720 (+ 384us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:38.243767 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:38.243797 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:38.244113 (+ 316us) tablet.cc:699] Row locks acquired
0808 13:35:38.244132 (+ 19us) write_op.cc:260] PREPARE: finished
0808 13:35:38.244257 (+ 125us) write_op.cc:270] Start()
0808 13:35:38.244356 (+ 99us) write_op.cc:276] Timestamp: P: 1786196138244190 usec, L: 0
0808 13:35:38.244371 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:38.244862 (+ 491us) log.cc:842] Serialized 759 byte log entry
0808 13:35:38.258289 (+ 13427us) op_driver.cc:444] REPLICATION: finished
0808 13:35:38.274812 (+ 16523us) write_op.cc:301] APPLY: starting
0808 13:35:38.274873 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:38.274977 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:38.275104 (+ 127us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:38.275394 (+ 290us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:38.275554 (+ 160us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:38.275602 (+ 48us) write_op.cc:312] APPLY: finished
0808 13:35:38.276215 (+ 613us) log.cc:842] Serialized 29 byte log entry
0808 13:35:38.276894 (+ 679us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.277046 (+ 152us) write_op.cc:454] Released schema lock
0808 13:35:38.277085 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:38.278653 (+ 1568us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.278683 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":4351232,"child_traces":[["op",{"apply.queue_time_us":16347,"apply.run_cpu_time_us":1848,"apply.run_wall_time_us":4097,"num_ops":2,"prepare.queue_time_us":4679,"prepare.run_cpu_time_us":2141,"prepare.run_wall_time_us":2141,"replication_time_us":13841}]]}
I20260808 13:35:39.063653 4839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1eb4e0870a824768a3ecace6bf383c25" } modify_external_catalogs: true
I20260808 13:35:39.065290 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85b4fd30d6c4903946891a399acf7a0" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:39.066469 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72c4ddec62c242c88b3a759d8361cd30" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:39.090407 4842 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fd822a88bb39430eb487b2bdc43e86fb" } new_table_name: "default.Table_811d61f013fe4c59853421dff8a641bb" modify_external_catalogs: true
I20260808 13:35:39.093685 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 896389941179456d9d0ed3cd14726f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:39.095968 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd6945ecccc4684a3fa5b2a1f07057f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:25 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:39.097749 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 69642fdba1f145438b04eae9f8154101 with delete_type TABLET_DATA_DELETED (replaced by tablet 538d9a883e9d46668fde30876352d20e) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:39.115237 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet d3682c9c656c4556bc13ae3aeddd7c5e with delete_type TABLET_DATA_DELETED (replaced by tablet ba39706cff65404485959865fff1772a) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:39.115358 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet e992a0294c404cfb9541b57782a05fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:39.116381 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 127e8d0323954ef788018bcb91582951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:46328
W20260808 13:35:39.117118 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 54) took 3653 ms (client timeout 1000 ms). Trace:
W20260808 13:35:39.121418 5551 leader_election.cc:341] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:39.122247 5551 leader_election.cc:341] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:39.130332 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:39.131489 7199 raft_consensus.cc:2258] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:38.996023 5772 consensus_peers.cc:487] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:39.133042 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 137096us
W20260808 13:35:39.143164 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 55) took 2606 ms (client timeout 1000 ms). Trace:
W20260808 13:35:39.149151 4837 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 56) took 1043 ms (client timeout 1000 ms). Trace:
W20260808 13:35:39.149670 4837 rpcz_store.cc:269] 0808 13:35:38.105431 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.111949 (+ 6518us) service_pool.cc:227] Handling call
0808 13:35:39.149077 (+1037128us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8382,"reported_tablets":31}
I20260808 13:35:39.152302 7013 ts_tablet_manager.cc:1918] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:39.152787 7013 log.cc:1196] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/1f9bc338093f403881d329a7e913c9f7
I20260808 13:35:39.156183 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85b4fd30d6c4903946891a399acf7a0" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:39.157624 5550 leader_election.cc:341] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:39.158032 5550 leader_election.cc:302] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:39.159026 7896 raft_consensus.cc:2726] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:39.160279 4836 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 39) took 2976 ms (client timeout 1000 ms). Trace:
I20260808 13:35:39.161026 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cd8f12cda94c4eb9a848f002a4020b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:37 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:39.160720 4836 rpcz_store.cc:269] 0808 13:35:36.183926 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.193324 (+ 9398us) service_pool.cc:227] Handling call
0808 13:35:39.160182 (+2966858us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":67886,"reported_tablets":28}
I20260808 13:35:39.164487 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:39.165154 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f2e0b98af431d99894e8778257bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:39.167459 5384 raft_consensus.cc:3037] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:39.045526 7918 rpcz_store.cc:276] 0808 13:35:37.789158 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:37.789371 (+ 213us) service_pool.cc:227] Handling call
0808 13:35:39.044964 (+1255593us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:37.797310 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:37.798117 (+ 807us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:37.798176 (+ 59us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:37.798288 (+ 112us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:37.798456 (+ 168us) alter_schema_op.cc:126] START. Timestamp: 1786196137798296
0808 13:35:37.798510 (+ 54us) op_driver.cc:331] REPLICATION: starting
0808 13:35:37.799549 (+ 1039us) log.cc:842] Serialized 173 byte log entry
0808 13:35:38.538244 (+738695us) op_driver.cc:444] REPLICATION: finished
0808 13:35:38.779884 (+241640us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:38.788832 (+ 8948us) log.cc:842] Serialized 5 byte log entry
0808 13:35:39.043082 (+254250us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.043961 (+ 879us) log.cc:842] Serialized 14 byte log entry
0808 13:35:39.044824 (+ 863us) alter_schema_op.cc:71] Released schema lock
0808 13:35:39.044848 (+ 24us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":241431,"mutex_wait_us":13389,"prepare.queue_time_us":6306,"prepare.run_cpu_time_us":2626,"prepare.run_wall_time_us":6337,"raft.queue_time_us":146,"raft.run_cpu_time_us":831,"raft.run_wall_time_us":830,"replication_time_us":739548,"thread_start_us":1124476,"threads_started":2}]]}
I20260808 13:35:39.175041 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72c4ddec62c242c88b3a759d8361cd30" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:39.175643 5875 raft_consensus.cc:3037] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:39.179157 4839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1eb4e0870a824768a3ecace6bf383c25" } modify_external_catalogs: true
I20260808 13:35:39.082933 7882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:39.179965 4840 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 421) took 1666 ms. Trace:
I20260808 13:35:39.180179 7882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e8b888041b4135af55792050defcb5, State: Initialized, Role: FOLLOWER
I20260808 13:35:39.181475 4837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_db9615359c8549588ff9f1d557189a0d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:39.183460 7947 tablet.cc:1722] T 4b7284afe0294644a368cf51451c7ab6 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
W20260808 13:35:39.143677 4835 rpcz_store.cc:269] 0808 13:35:36.536478 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.539703 (+ 3225us) service_pool.cc:227] Handling call
0808 13:35:38.034726 (+1495023us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.034760 (+ 34us) write_op.cc:454] Released schema lock
0808 13:35:39.143091 (+1108331us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:38.013321 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:38.013819 (+ 498us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:38.013854 (+ 35us) write_op.cc:435] Acquired schema lock
0808 13:35:38.013874 (+ 20us) tablet.cc:660] Decoding operations
0808 13:35:38.014215 (+ 341us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:38.014278 (+ 63us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:38.014310 (+ 32us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:38.014487 (+ 177us) tablet.cc:699] Row locks acquired
0808 13:35:38.014506 (+ 19us) write_op.cc:260] PREPARE: finished
0808 13:35:38.014660 (+ 154us) write_op.cc:270] Start()
0808 13:35:38.014761 (+ 101us) write_op.cc:276] Timestamp: P: 1786196138014591 usec, L: 0
0808 13:35:38.014783 (+ 22us) op_driver.cc:331] REPLICATION: starting
0808 13:35:38.015418 (+ 635us) log.cc:842] Serialized 356 byte log entry
0808 13:35:38.025263 (+ 9845us) op_driver.cc:444] REPLICATION: finished
0808 13:35:38.025737 (+ 474us) write_op.cc:301] APPLY: starting
0808 13:35:38.025814 (+ 77us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:38.025932 (+ 118us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:38.026051 (+ 119us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:38.026284 (+ 233us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:38.026463 (+ 179us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:38.026525 (+ 62us) write_op.cc:312] APPLY: finished
0808 13:35:38.026747 (+ 222us) log.cc:842] Serialized 23 byte log entry
0808 13:35:38.026924 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.027012 (+ 88us) write_op.cc:454] Released schema lock
0808 13:35:38.027052 (+ 40us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":33347,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":310,"apply.run_cpu_time_us":1548,"apply.run_wall_time_us":1546,"num_ops":1,"prepare.queue_time_us":5711,"prepare.run_cpu_time_us":2501,"prepare.run_wall_time_us":2501,"raft.queue_time_us":559,"raft.run_cpu_time_us":833,"raft.run_wall_time_us":832,"replication_time_us":10369}]]}
I20260808 13:35:39.186811 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 9423b380a62c467895e7fdff14756586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:39.188685 7918 tablet.cc:1722] T b1847d0fecb843cdbc833ad674a503a4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:39.191881 7906 ts_tablet_manager.cc:1918] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:39.194487 6033 raft_consensus.cc:348] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.195222 6033 raft_consensus.cc:374] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:39.195464 6033 raft_consensus.cc:729] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:39.196084 6033 consensus_queue.cc:260] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.198555 6033 ts_tablet_manager.cc:1428] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.260s user 0.006s sys 0.000s
I20260808 13:35:39.199383 6033 tablet_bootstrap.cc:492] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:39.201150 7199 ts_tablet_manager.cc:1905] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:39.201658 7882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
W20260808 13:35:39.206593 4837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db9615359c8549588ff9f1d557189a0d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:39.217914 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 84054.5us
I20260808 13:35:39.223085 7906 log.cc:1196] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/328d361c336848689c6906ca6fe2ccc3
W20260808 13:35:39.227118 4838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 40) took 1937 ms (client timeout 1000 ms). Trace:
I20260808 13:35:39.227494 5875 raft_consensus.cc:2454] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
W20260808 13:35:39.227521 4838 rpcz_store.cc:269] 0808 13:35:37.289911 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:37.302208 (+ 12297us) service_pool.cc:227] Handling call
0808 13:35:39.227051 (+1924843us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:38.187301 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:38.187627 (+ 326us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:38.187648 (+ 21us) write_op.cc:435] Acquired schema lock
0808 13:35:38.187659 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:38.187903 (+ 244us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:38.187937 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:38.187957 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:38.188079 (+ 122us) tablet.cc:699] Row locks acquired
0808 13:35:38.188092 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:38.188186 (+ 94us) write_op.cc:270] Start()
0808 13:35:38.188255 (+ 69us) write_op.cc:276] Timestamp: P: 1786196138188131 usec, L: 0
0808 13:35:38.188270 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:38.190117 (+ 1847us) log.cc:842] Serialized 527 byte log entry
0808 13:35:38.203355 (+ 13238us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b58000c05b0. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c1a9834 00007f227c1ae771 00007f227c1ae6f9 00007f227c1ae689 00007f227c1ae651 00007f227c1ad81d 0000000000580784 00000000005805b9 00007f22752da6b6 00007f22752ddb11
0808 13:35:38.203815 (+ 460us) op_driver.cc:444] REPLICATION: finished
0808 13:35:38.212755 (+ 8940us) write_op.cc:301] APPLY: starting
0808 13:35:38.212822 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:38.212918 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:38.212942 (+ 24us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:38.213100 (+ 158us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:38.213233 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:38.213278 (+ 45us) write_op.cc:312] APPLY: finished
0808 13:35:38.213446 (+ 168us) log.cc:842] Serialized 23 byte log entry
0808 13:35:38.213589 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.213660 (+ 71us) write_op.cc:454] Released schema lock
0808 13:35:38.213693 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:38.235546 (+ 21853us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.235573 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":8786,"apply.run_cpu_time_us":1347,"apply.run_wall_time_us":22983,"mutex_wait_us":41174,"num_ops":1,"prepare.queue_time_us":43508,"prepare.run_cpu_time_us":1675,"prepare.run_wall_time_us":3057,"raft.queue_time_us":155,"raft.run_cpu_time_us":814,"raft.run_wall_time_us":12953,"replication_time_us":15487,"spinlock_wait_cycles":28469504,"thread_start_us":16720,"threads_started":1}]]}
I20260808 13:35:39.231737 7906 ts_tablet_manager.cc:1939] T 328d361c336848689c6906ca6fe2ccc3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
W20260808 13:35:39.237178 7944 log.cc:924] Time spent T 16a6552c871b4b81b6520cb084d359a2 P 946c9d8462644deba5b45fa724f1923c: Append to log took a long time: real 0.055s user 0.000s sys 0.001s
I20260808 13:35:39.197912 6026 tablet_bootstrap.cc:492] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:39.239018 6026 ts_tablet_manager.cc:1397] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.419s user 0.011s sys 0.012s
I20260808 13:35:39.241739 6026 raft_consensus.cc:348] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.242391 6026 raft_consensus.cc:374] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:39.243753 6026 raft_consensus.cc:729] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:39.249586 6026 consensus_queue.cc:260] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.250751 7013 ts_tablet_manager.cc:1939] T 1f9bc338093f403881d329a7e913c9f7 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:39.251627 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 366) took 3515 ms. Trace:
I20260808 13:35:39.251978 7013 rpcz_store.cc:276] 0808 13:35:35.735791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.742129 (+ 6338us) service_pool.cc:227] Handling call
0808 13:35:38.826249 (+3084120us) ts_tablet_manager.cc:1145] Deleting tablet 1f9bc338093f403881d329a7e913c9f7
0808 13:35:38.826288 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:38.954599 (+128311us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.100940 (+146341us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.251572 (+150632us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3083594}
I20260808 13:35:39.252480 7906 tablet_replica.cc:330] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:39.253159 7013 tablet_replica.cc:330] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:39.253952 7013 raft_consensus.cc:2229] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:39.254427 7013 raft_consensus.cc:2258] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:39.257158 7013 ts_tablet_manager.cc:1905] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:39.260160 6046 tablet_bootstrap.cc:654] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:39.267217 6026 ts_tablet_manager.cc:1428] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.028s user 0.006s sys 0.001s
I20260808 13:35:39.268652 7906 raft_consensus.cc:2229] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:39.271369 7906 raft_consensus.cc:2258] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:39.287626 7906 ts_tablet_manager.cc:1905] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:39.300604 7882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:35:39.294458 7961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:35:39.302865 7956 tablet.cc:1722] T b1847d0fecb843cdbc833ad674a503a4 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260808 13:35:39.117470 4832 rpcz_store.cc:269] 0808 13:35:35.463249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.562277 (+ 99028us) service_pool.cc:227] Handling call
0808 13:35:39.117044 (+3554767us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:36.804446 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:36.804923 (+ 477us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:36.804947 (+ 24us) write_op.cc:435] Acquired schema lock
0808 13:35:36.804959 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:36.805227 (+ 268us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:36.805338 (+ 111us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:36.805362 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:36.805510 (+ 148us) tablet.cc:699] Row locks acquired
0808 13:35:36.805523 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:36.805624 (+ 101us) write_op.cc:270] Start()
0808 13:35:36.805802 (+ 178us) write_op.cc:276] Timestamp: P: 1786196136805552 usec, L: 0
0808 13:35:36.805818 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:37.039878 (+234060us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b5400022018. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227cf570c4 00007f227c2034f9 00007f227c1bef91 00007f227c1bec6a 00007f227c22acd5 00007f227c22b179 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9
0808 13:35:37.040206 (+ 328us) log.cc:842] Serialized 527 byte log entry
0808 13:35:37.064051 (+ 23845us) op_driver.cc:444] REPLICATION: finished
0808 13:35:37.064582 (+ 531us) write_op.cc:301] APPLY: starting
0808 13:35:37.064631 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:37.064723 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:37.064742 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:37.064935 (+ 193us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:37.065057 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:37.065102 (+ 45us) write_op.cc:312] APPLY: finished
0808 13:35:37.065280 (+ 178us) log.cc:842] Serialized 23 byte log entry
0808 13:35:37.065417 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.065474 (+ 57us) write_op.cc:454] Released schema lock
0808 13:35:37.065506 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:37.065664 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.065686 (+ 22us) write_op.cc:454] Released schema lock
0808 13:35:37.143711 (+ 78025us) spinlock_profiling.cc:243] Waited 98.7 ms on lock 0x7b58000c05b0. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c1a9834 00007f227c1ae771 00007f227c1ae6f9 00007f227c1ae689 00007f227c1ae651 00007f227c1ad81d 0000000000580784 00000000005805b9 00007f22752da6b6 00007f22752ddb11
Related trace 'op':
0808 13:35:37.923020 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:37.923511 (+ 491us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:37.923546 (+ 35us) write_op.cc:435] Acquired schema lock
0808 13:35:37.923567 (+ 21us) tablet.cc:660] Decoding operations
0808 13:35:37.923928 (+ 361us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:37.923985 (+ 57us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:37.924020 (+ 35us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:37.924207 (+ 187us) tablet.cc:699] Row locks acquired
0808 13:35:37.924227 (+ 20us) write_op.cc:260] PREPARE: finished
0808 13:35:37.924352 (+ 125us) write_op.cc:270] Start()
0808 13:35:37.924458 (+ 106us) write_op.cc:276] Timestamp: P: 1786196137924252 usec, L: 0
0808 13:35:37.924483 (+ 25us) op_driver.cc:331] REPLICATION: starting
0808 13:35:37.925006 (+ 523us) log.cc:842] Serialized 356 byte log entry
0808 13:35:37.936134 (+ 11128us) op_driver.cc:444] REPLICATION: finished
0808 13:35:37.939073 (+ 2939us) write_op.cc:301] APPLY: starting
0808 13:35:37.939144 (+ 71us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:37.939280 (+ 136us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:37.939392 (+ 112us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:37.939610 (+ 218us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:37.939776 (+ 166us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:37.939839 (+ 63us) write_op.cc:312] APPLY: finished
0808 13:35:37.940052 (+ 213us) log.cc:842] Serialized 23 byte log entry
0808 13:35:37.940224 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.940312 (+ 88us) write_op.cc:454] Released schema lock
0808 13:35:37.940340 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:37.940508 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:37.940529 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":51520,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":380,"apply.run_cpu_time_us":1249,"apply.run_wall_time_us":1248,"num_ops":1,"prepare.queue_time_us":299,"prepare.run_cpu_time_us":5398,"prepare.run_wall_time_us":244361,"raft.queue_time_us":4037,"raft.run_cpu_time_us":2830,"raft.run_wall_time_us":102023,"replication_time_us":258132,"spinlock_wait_cycles":760987392,"thread_start_us":8183,"threads_started":1}],["op",{"apply.queue_time_us":2725,"apply.run_cpu_time_us":1629,"apply.run_wall_time_us":1628,"num_ops":1,"prepare.queue_time_us":122,"prepare.run_cpu_time_us":2328,"prepare.run_wall_time_us":2330,"raft.queue_time_us":2536,"raft.run_cpu_time_us":700,"raft.run_wall_time_us":738,"replication_time_us":11559}]]}
I20260808 13:35:39.239500 4833 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 422) took 1621 ms. Trace:
I20260808 13:35:39.180262 4840 rpcz_store.cc:276] 0808 13:35:37.513568 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.491351 (+977783us) service_pool.cc:227] Handling call
0808 13:35:38.518940 (+ 27589us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:38.522720 (+ 3780us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:38.523275 (+ 555us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:38.523472 (+ 197us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:38.565669 (+ 42197us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:39.166806 (+601137us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:38.524767 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:38.525161 (+ 394us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:38.525183 (+ 22us) write_op.cc:435] Acquired schema lock
0808 13:35:38.525203 (+ 20us) tablet.cc:660] Decoding operations
0808 13:35:38.525455 (+ 252us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:38.525496 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:38.525522 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:38.525656 (+ 134us) tablet.cc:699] Row locks acquired
0808 13:35:38.525669 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:38.525775 (+ 106us) write_op.cc:270] Start()
0808 13:35:38.525852 (+ 77us) write_op.cc:276] Timestamp: P: 1786196138525736 usec, L: 0
0808 13:35:38.525868 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:38.526419 (+ 551us) log.cc:842] Serialized 405 byte log entry
0808 13:35:38.541533 (+ 15114us) op_driver.cc:444] REPLICATION: finished
0808 13:35:38.541882 (+ 349us) write_op.cc:301] APPLY: starting
0808 13:35:38.541936 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:38.542026 (+ 90us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:38.542131 (+ 105us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:38.542288 (+ 157us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:38.542423 (+ 135us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:38.542467 (+ 44us) write_op.cc:312] APPLY: finished
0808 13:35:38.542681 (+ 214us) log.cc:842] Serialized 23 byte log entry
0808 13:35:38.542851 (+ 170us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.542918 (+ 67us) write_op.cc:454] Released schema lock
0808 13:35:38.542945 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:38.543123 (+ 178us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:38.543152 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":185,"apply.run_cpu_time_us":1473,"apply.run_wall_time_us":1472,"num_ops":1,"prepare.queue_time_us":129,"prepare.run_cpu_time_us":1989,"prepare.run_wall_time_us":4654,"raft.queue_time_us":2893,"raft.run_cpu_time_us":636,"raft.run_wall_time_us":634,"replication_time_us":15587}]]}
I20260808 13:35:39.205031 4835 master_service.cc:946] ReplaceTablet: received request to replace tablet 107309c7ada246b6a15a784d7a8c399d from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:39.245070 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 328d361c336848689c6906ca6fe2ccc3 (table default.Table_b23ea067bd8149539074d42c1596a2aa [id=3a77dc355690462ba633f54fadbeb544]) successfully deleted
W20260808 13:35:39.321547 4821 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 76490.3us
W20260808 13:35:39.323482 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 1f9bc338093f403881d329a7e913c9f7 because the tablet was not found. No further retry: Not found: Tablet not found: 1f9bc338093f403881d329a7e913c9f7
I20260808 13:35:39.325446 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_e9d344caa3c2448784d6108b083eca10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:39.320220 4833 rpcz_store.cc:276] 0808 13:35:37.618542 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.028484 (+1409942us) service_pool.cc:227] Handling call
0808 13:35:39.166195 (+137711us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:39.211720 (+ 45525us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:39.238677 (+ 26957us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:39.238702 (+ 25us) write_op.cc:454] Released schema lock
0808 13:35:39.238891 (+ 189us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:39.239181 (+ 290us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:39.239426 (+ 245us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:39.213056 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:39.213420 (+ 364us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:39.213443 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:39.213455 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:39.213722 (+ 267us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:39.213782 (+ 60us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:39.213804 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:39.214000 (+ 196us) tablet.cc:699] Row locks acquired
0808 13:35:39.214013 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:39.214120 (+ 107us) write_op.cc:270] Start()
0808 13:35:39.214198 (+ 78us) write_op.cc:276] Timestamp: P: 1786196139214062 usec, L: 0
0808 13:35:39.214213 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:39.214699 (+ 486us) log.cc:842] Serialized 433 byte log entry
0808 13:35:39.222748 (+ 8049us) op_driver.cc:444] REPLICATION: finished
0808 13:35:39.223073 (+ 325us) write_op.cc:301] APPLY: starting
0808 13:35:39.223171 (+ 98us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:39.223279 (+ 108us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:39.223359 (+ 80us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:39.223735 (+ 376us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:39.223873 (+ 138us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:39.223914 (+ 41us) write_op.cc:312] APPLY: finished
0808 13:35:39.224078 (+ 164us) log.cc:842] Serialized 29 byte log entry
0808 13:35:39.224206 (+ 128us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:39.224281 (+ 75us) write_op.cc:454] Released schema lock
0808 13:35:39.224318 (+ 37us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":192,"apply.run_cpu_time_us":1417,"apply.run_wall_time_us":1416,"num_ops":2,"prepare.queue_time_us":93,"prepare.run_cpu_time_us":1899,"prepare.run_wall_time_us":1900,"raft.queue_time_us":1146,"raft.run_cpu_time_us":635,"raft.run_wall_time_us":636,"replication_time_us":8466}]]}
I20260808 13:35:39.331024 4838 master_service.cc:946] ReplaceTablet: received request to replace tablet fc9b2e69fe27431dac9eb5ecb9e3e23e from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:39.332365 7908 mvcc.cc:205] Tried to move back new op lower bound from 7316259380421820416 to 7316259374004875264. Current Snapshot: MvccSnapshot[applied={T|T < 7316259380421820416}]
I20260808 13:35:39.248569 5772 leader_election.cc:302] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:39.331524 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_642fc6953dc8471cbd814ae6fda0cc6a" } modify_external_catalogs: true
I20260808 13:35:39.333949 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_642fc6953dc8471cbd814ae6fda0cc6a" } modify_external_catalogs: true
W20260808 13:35:39.334062 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 85572.2us
W20260808 13:35:39.335009 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9d344caa3c2448784d6108b083eca10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:39.275924 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 1f9bc338093f403881d329a7e913c9f7 (table default.Table_49299116c37c4b9dbb509dbc8efae85e [id=d88dc1106b9b4e1eb18dc680d15fdfcf]) successfully deleted
I20260808 13:35:39.279503 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet d0767140736242f18ef4689b112deefb
I20260808 13:35:39.339109 4839 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 423) took 1222 ms. Trace:
I20260808 13:35:39.339411 4839 rpcz_store.cc:276] 0808 13:35:38.117133 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.063317 (+946184us) service_pool.cc:227] Handling call
0808 13:35:39.179717 (+116400us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:39.236228 (+ 56511us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:39.236360 (+ 132us) catalog_manager.cc:2761] Locking tablets
0808 13:35:39.236931 (+ 571us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:39.278694 (+ 41763us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:39.278857 (+ 163us) write_op.cc:454] Released schema lock
0808 13:35:39.279088 (+ 231us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:39.279317 (+ 229us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:39.279423 (+ 106us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:39.339053 (+ 59630us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:39.257080 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:39.257458 (+ 378us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:39.257483 (+ 25us) write_op.cc:435] Acquired schema lock
0808 13:35:39.257494 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:39.257815 (+ 321us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:39.257856 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:39.257876 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:39.258168 (+ 292us) tablet.cc:699] Row locks acquired
0808 13:35:39.258182 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:39.258327 (+ 145us) write_op.cc:270] Start()
0808 13:35:39.258410 (+ 83us) write_op.cc:276] Timestamp: P: 1786196139258229 usec, L: 0
0808 13:35:39.258425 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:39.258926 (+ 501us) log.cc:842] Serialized 725 byte log entry
0808 13:35:39.266932 (+ 8006us) op_driver.cc:444] REPLICATION: finished
0808 13:35:39.267226 (+ 294us) write_op.cc:301] APPLY: starting
0808 13:35:39.267288 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:39.267395 (+ 107us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:39.267477 (+ 82us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:39.267713 (+ 236us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:39.267870 (+ 157us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:39.267908 (+ 38us) write_op.cc:312] APPLY: finished
0808 13:35:39.268079 (+ 171us) log.cc:842] Serialized 29 byte log entry
0808 13:35:39.268208 (+ 129us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:39.268284 (+ 76us) write_op.cc:454] Released schema lock
0808 13:35:39.268316 (+ 32us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":157,"apply.run_cpu_time_us":1271,"apply.run_wall_time_us":1269,"num_ops":2,"prepare.queue_time_us":91,"prepare.run_cpu_time_us":2126,"prepare.run_wall_time_us":2126,"raft.queue_time_us":1375,"raft.run_cpu_time_us":586,"raft.run_wall_time_us":587,"replication_time_us":8430}]]}
I20260808 13:35:39.210372 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_cfa1c9cda5c140d2920999367a8c9509"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:39.343542 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cfa1c9cda5c140d2920999367a8c9509 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:39.344426 7915 raft_consensus.cc:2726] T 72c4ddec62c242c88b3a759d8361cd30 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:39.344775 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet d8b07b7d31ef41e484389c0f007598f0 (table default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d8b07b7d31ef41e484389c0f007598f0 consensus state has no leader
W20260808 13:35:39.294961 4841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 41) took 945 ms (client timeout 1000 ms). Trace:
W20260808 13:35:39.346328 4841 rpcz_store.cc:269] 0808 13:35:38.349071 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.446264 (+ 97193us) service_pool.cc:227] Handling call
0808 13:35:39.294897 (+848633us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":29}
I20260808 13:35:39.346460 5550 leader_election.cc:302] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:39.348073 4820 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 72167.1us
I20260808 13:35:39.349632 7690 raft_consensus.cc:2781] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:39.351609 7690 raft_consensus.cc:686] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:39.352579 7690 consensus_queue.cc:237] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.353782 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet d0767140736242f18ef4689b112deefb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:39.355486 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet d0767140736242f18ef4689b112deefb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:39.337965 7956 tablet.cc:1722] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260808 13:35:39.361321 4842 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b8e3245c8ba54f69bc329a5b75115127 consensus state has no leader
I20260808 13:35:39.361816 4842 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 424) took 1048 ms. Trace:
I20260808 13:35:39.362073 4842 rpcz_store.cc:276] 0808 13:35:38.313225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.090046 (+776821us) service_pool.cc:227] Handling call
0808 13:35:39.240597 (+150551us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5400003110. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227511ef11 00007f22752a3320 00007f227e0e44ca 00007f227e1ff812 00007f227bd45634 00007f227bd455c2 00007f227bd454f1 00007f227bd4546c 00007f227bd44722 00007f2279fde94c 00007f2279fddd86
0808 13:35:39.240664 (+ 67us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:39.241295 (+ 631us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:39.241838 (+ 543us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:39.242035 (+ 197us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:39.360735 (+118700us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:39.361764 (+ 1029us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:39.285228 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:39.285596 (+ 368us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:39.285619 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:39.285631 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:39.285891 (+ 260us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:39.285936 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:39.285955 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:39.286102 (+ 147us) tablet.cc:699] Row locks acquired
0808 13:35:39.286116 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:39.286217 (+ 101us) write_op.cc:270] Start()
0808 13:35:39.286304 (+ 87us) write_op.cc:276] Timestamp: P: 1786196139286152 usec, L: 0
0808 13:35:39.286321 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:39.286826 (+ 505us) log.cc:842] Serialized 405 byte log entry
0808 13:35:39.358228 (+ 71402us) op_driver.cc:444] REPLICATION: finished
0808 13:35:39.358546 (+ 318us) write_op.cc:301] APPLY: starting
0808 13:35:39.358620 (+ 74us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:39.358737 (+ 117us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:39.358856 (+ 119us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:39.359059 (+ 203us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:39.359233 (+ 174us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:39.359304 (+ 71us) write_op.cc:312] APPLY: finished
0808 13:35:39.359478 (+ 174us) log.cc:842] Serialized 23 byte log entry
0808 13:35:39.359625 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:39.359685 (+ 60us) write_op.cc:454] Released schema lock
0808 13:35:39.359714 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:39.359890 (+ 176us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:39.359916 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":25668480,"child_traces":[["op",{"apply.queue_time_us":174,"apply.run_cpu_time_us":1515,"apply.run_wall_time_us":1514,"num_ops":1,"prepare.queue_time_us":93,"prepare.run_cpu_time_us":1851,"prepare.run_wall_time_us":1861,"raft.queue_time_us":1114,"raft.run_cpu_time_us":508,"raft.run_wall_time_us":507,"replication_time_us":71832}]]}
I20260808 13:35:39.367183 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet d0767140736242f18ef4689b112deefb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:39.312394 7961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:35:39.376456 6046 tablet_bootstrap.cc:492] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:39.376961 6046 ts_tablet_manager.cc:1397] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.442s user 0.003s sys 0.010s
I20260808 13:35:39.290426 7935 tablet.cc:1722] T b1847d0fecb843cdbc833ad674a503a4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260808 13:35:39.293171 5883 thread.cc:636] apply [worker] (thread pool) Time spent creating pthread: real 0.515s user 0.016s sys 0.039s
I20260808 13:35:39.377477 6026 tablet_bootstrap.cc:492] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:39.377714 5883 thread.cc:602] apply [worker] (thread pool) Time spent starting thread: real 0.600s user 0.016s sys 0.039s
I20260808 13:35:39.379155 6046 raft_consensus.cc:348] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.330036 7937 tablet.cc:1722] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:39.384712 6026 tablet_bootstrap.cc:654] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:39.382978 7918 tablet.cc:1722] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:39.414803 6033 tablet_bootstrap.cc:654] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:39.419349 6046 raft_consensus.cc:374] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:39.419669 6046 raft_consensus.cc:729] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:35:39.420583 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 79862.8us
I20260808 13:35:39.376356 7975 tablet.cc:1722] T 2bb8c7eeec164485a4e5bb80f9d5fd72 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:39.452791 7957 consensus_queue.cc:1035] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:39.463699 6026 tablet_bootstrap.cc:492] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:39.464141 6026 ts_tablet_manager.cc:1397] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.087s user 0.008s sys 0.003s
I20260808 13:35:39.465911 7915 consensus_queue.cc:1035] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:35:39.467134 7976 log.cc:924] Time spent T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.073s user 0.000s sys 0.001s
I20260808 13:35:39.469396 7915 raft_consensus.cc:482] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:39.470110 7915 raft_consensus.cc:504] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.475371 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adf9a03742a14cfe867c578c466838de" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:39.476609 5771 leader_election.cc:341] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:39.481890 6046 consensus_queue.cc:260] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.484704 6046 ts_tablet_manager.cc:1428] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.107s user 0.005s sys 0.000s
I20260808 13:35:39.516683 6026 raft_consensus.cc:348] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.517568 6026 raft_consensus.cc:374] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:39.517891 6026 raft_consensus.cc:729] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:39.518720 6026 consensus_queue.cc:260] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.522390 7915 leader_election.cc:288] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:39.525213 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 6e3ad9ca75b541798bc04d7701128faa
I20260808 13:35:39.528852 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e3ad9ca75b541798bc04d7701128faa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:39.529495 7906 ts_tablet_manager.cc:1918] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:39.551782 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e3ad9ca75b541798bc04d7701128faa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:39.530524 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e3ad9ca75b541798bc04d7701128faa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:39.532429 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adf9a03742a14cfe867c578c466838de" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:39.572645 7906 log.cc:1196] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/264ee45f8dfa4a05b0e5e812ce1b622c
I20260808 13:35:39.572886 5630 raft_consensus.cc:2454] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:39.574002 7906 ts_tablet_manager.cc:1939] T 264ee45f8dfa4a05b0e5e812ce1b622c P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:39.555123 6026 ts_tablet_manager.cc:1428] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.091s user 0.008s sys 0.001s
I20260808 13:35:39.575456 6026 tablet_bootstrap.cc:492] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:39.576623 5772 leader_election.cc:302] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:39.580118 7973 raft_consensus.cc:2781] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:39.580870 7973 raft_consensus.cc:482] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:39.581692 7973 raft_consensus.cc:3037] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:39.591845 6046 tablet_bootstrap.cc:492] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:39.604913 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_f6d782e61daf4dd5be5685aea75e9fd2" } new_table_name: "default.Table_9f6ee103b59c4a17b37f857fbb83c22c" modify_external_catalogs: true
I20260808 13:35:39.611833 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_f3144416bf1140aaaa78d9ec41319c6b" } modify_external_catalogs: true
I20260808 13:35:39.612375 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_f3144416bf1140aaaa78d9ec41319c6b" } modify_external_catalogs: true
I20260808 13:35:39.619063 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 371) took 1180 ms. Trace:
I20260808 13:35:39.619417 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_a4c8baa577ff42f4b8f9aafb6ec9e6ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:39.620350 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_4ef3501cc89b4779b73a48142fd01d36" } new_table_name: "default.Table_a1db036771e0467fb083f890fefd54e9" modify_external_catalogs: true
W20260808 13:35:39.620929 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4c8baa577ff42f4b8f9aafb6ec9e6ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:39.622730 7906 rpcz_store.cc:276] 0808 13:35:38.439101 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.439482 (+ 381us) service_pool.cc:227] Handling call
0808 13:35:39.252318 (+812836us) ts_tablet_manager.cc:1145] Deleting tablet 264ee45f8dfa4a05b0e5e812ce1b622c
0808 13:35:39.252359 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:39.372045 (+119686us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.529222 (+157177us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.618964 (+ 89742us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3938,"tablet-delete.queue_time_us":812162}
I20260808 13:35:39.631006 7961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:39.631775 7961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:39.640085 7961 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:39.643535 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 372) took 1201 ms. Trace:
I20260808 13:35:39.645648 7906 rpcz_store.cc:276] 0808 13:35:38.439467 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.440311 (+ 844us) service_pool.cc:227] Handling call
0808 13:35:39.640126 (+1199815us) ts_tablet_manager.cc:1145] Deleting tablet 9423b380a62c467895e7fdff14756586
0808 13:35:39.640186 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:39.640452 (+ 266us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1199269}
I20260808 13:35:39.645387 6026 tablet_bootstrap.cc:654] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:39.571956 7995 tablet.cc:1722] T fcf907a9f19f4d64b27be946011a8ddb P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:39.662022 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:39.664048 7013 ts_tablet_manager.cc:1918] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:39.667233 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 373) took 1227 ms. Trace:
I20260808 13:35:39.673333 7906 rpcz_store.cc:276] 0808 13:35:38.439794 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.512138 (+ 72344us) service_pool.cc:227] Handling call
0808 13:35:39.661513 (+1149375us) ts_tablet_manager.cc:1145] Deleting tablet 7e8cf06462484c99a860762a82f7ad64
0808 13:35:39.666742 (+ 5229us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:39.667148 (+ 406us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1107100}
I20260808 13:35:39.699692 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 370) took 1257 ms. Trace:
I20260808 13:35:39.626528 7997 tablet.cc:1722] T fcf907a9f19f4d64b27be946011a8ddb P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:39.701231 7906 rpcz_store.cc:276] 0808 13:35:38.438709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.512083 (+ 73374us) service_pool.cc:227] Handling call
0808 13:35:39.694437 (+1182354us) ts_tablet_manager.cc:1145] Deleting tablet fc9b2e69fe27431dac9eb5ecb9e3e23e
0808 13:35:39.694493 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:39.696485 (+ 1992us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1139436}
I20260808 13:35:39.702832 7013 log.cc:1196] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/fc9b2e69fe27431dac9eb5ecb9e3e23e
I20260808 13:35:39.659489 7199 ts_tablet_manager.cc:1918] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:39.703806 7013 ts_tablet_manager.cc:1939] T fc9b2e69fe27431dac9eb5ecb9e3e23e P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:39.703908 7199 log.cc:1196] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/69642fdba1f145438b04eae9f8154101
I20260808 13:35:39.704721 7199 ts_tablet_manager.cc:1939] T 69642fdba1f145438b04eae9f8154101 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:39.706372 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 373) took 3604 ms. Trace:
I20260808 13:35:39.711740 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:39.713196 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet b1847d0fecb843cdbc833ad674a503a4 (table default.Table_2d75fe6a74ad43789e515c67b5167ab3 [id=29db8794594e4221be8974258b5127e0]). Expected version 1 got 0
I20260808 13:35:39.706689 7013 rpcz_store.cc:276] 0808 13:35:36.101713 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.138207 (+ 36494us) service_pool.cc:227] Handling call
0808 13:35:39.253021 (+3114814us) ts_tablet_manager.cc:1145] Deleting tablet fc9b2e69fe27431dac9eb5ecb9e3e23e
0808 13:35:39.253056 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:39.381732 (+128676us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.663836 (+282104us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.706313 (+ 42477us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3112979}
I20260808 13:35:39.715890 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:39.717250 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 374) took 3559 ms. Trace:
W20260808 13:35:39.699864 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0329c348c88a4ddda0f2b90be7a4af76 consensus state has no leader
W20260808 13:35:39.697464 4833 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 52deb2e968c04b99a436e04ae78d9969 consensus state has no leader
I20260808 13:35:39.717298 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet fc9b2e69fe27431dac9eb5ecb9e3e23e (table default.Table_13f833fb48504c76865676e23461e6e5 [id=0a8dbdef483f48b68019ea98b335a11e]) successfully deleted
I20260808 13:35:39.720208 7013 rpcz_store.cc:276] 0808 13:35:36.157930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.158062 (+ 132us) service_pool.cc:227] Handling call
0808 13:35:39.716931 (+3558869us) ts_tablet_manager.cc:1145] Deleting tablet e5fb1d373c4d486ca3815023897721a2
0808 13:35:39.716971 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:39.717195 (+ 224us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3558067}
I20260808 13:35:39.721736 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:39.724016 7013 tablet_replica.cc:330] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:39.698208 4836 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 427) took 1043 ms. Trace:
W20260808 13:35:39.723268 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet e5fb1d373c4d486ca3815023897721a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5fb1d373c4d486ca3815023897721a2 already in progress: creating tablet
I20260808 13:35:39.727341 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet d8b07b7d31ef41e484389c0f007598f0 (table default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]). Expected version 1 got 0
I20260808 13:35:39.728410 7013 raft_consensus.cc:2229] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:39.726464 4836 rpcz_store.cc:276] 0808 13:35:38.654367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.209966 (+555599us) service_pool.cc:227] Handling call
0808 13:35:39.343970 (+134004us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:39.423674 (+ 79704us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:39.658465 (+234791us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:39.697832 (+ 39367us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:39.698144 (+ 312us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:39.514508 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:39.514961 (+ 453us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:39.514983 (+ 22us) write_op.cc:435] Acquired schema lock
0808 13:35:39.514994 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:39.515251 (+ 257us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:39.515285 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:39.515306 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:39.515504 (+ 198us) tablet.cc:699] Row locks acquired
0808 13:35:39.515516 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:39.515625 (+ 109us) write_op.cc:270] Start()
0808 13:35:39.515708 (+ 83us) write_op.cc:276] Timestamp: P: 1786196139515528 usec, L: 0
0808 13:35:39.515723 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:39.516202 (+ 479us) log.cc:842] Serialized 433 byte log entry
0808 13:35:39.614628 (+ 98426us) op_driver.cc:444] REPLICATION: finished
0808 13:35:39.649663 (+ 35035us) write_op.cc:301] APPLY: starting
0808 13:35:39.649725 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:39.649849 (+ 124us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:39.650006 (+ 157us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:39.650497 (+ 491us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:39.656735 (+ 6238us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:39.657555 (+ 820us) write_op.cc:312] APPLY: finished
0808 13:35:39.657896 (+ 341us) log.cc:842] Serialized 29 byte log entry
0808 13:35:39.658110 (+ 214us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:39.658189 (+ 79us) write_op.cc:454] Released schema lock
0808 13:35:39.658251 (+ 62us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:39.658465 (+ 214us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:39.658498 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":34834,"apply.run_cpu_time_us":2433,"apply.run_wall_time_us":48769,"mutex_wait_us":22697,"num_ops":2,"prepare.queue_time_us":89703,"prepare.run_cpu_time_us":1864,"prepare.run_wall_time_us":1864,"replication_time_us":98812}]]}
I20260808 13:35:39.730147 7013 raft_consensus.cc:2258] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:39.730511 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 2bb8c7eeec164485a4e5bb80f9d5fd72 (table default.Table_62ad61e62a5b49b480ec08bc990efa85 [id=8e1d239d7f7446fa89184bf11ddb88d9]). Expected version 1 got 0
I20260808 13:35:39.667156 6033 tablet_bootstrap.cc:492] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:39.733515 6033 ts_tablet_manager.cc:1397] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.534s user 0.007s sys 0.008s
I20260808 13:35:39.735165 7013 ts_tablet_manager.cc:1905] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:39.733008 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:39.736624 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:39.737862 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f85b4fd30d6c4903946891a399acf7a0 (table default.Table_7da887b39f244cf0b44cbe9fa3b0eae2 [id=ea2ba343c17d403d8cb0bf9d80bee453]). Expected version 1 got 0
I20260808 13:35:39.742193 7975 tablet.cc:1722] T fcf907a9f19f4d64b27be946011a8ddb P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:39.709247 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 355) took 4148 ms. Trace:
I20260808 13:35:39.743726 7199 rpcz_store.cc:276] 0808 13:35:35.556904 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.557042 (+ 138us) service_pool.cc:227] Handling call
0808 13:35:39.013547 (+3456505us) ts_tablet_manager.cc:1145] Deleting tablet 69642fdba1f145438b04eae9f8154101
0808 13:35:39.013590 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:39.287674 (+274084us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.562573 (+274899us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.705577 (+143004us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":109292,"tablet-delete.queue_time_us":3455956}
I20260808 13:35:39.745522 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 69642fdba1f145438b04eae9f8154101 (table default.Table_41f3478fe8c94fbfaf7fcbdee22ce987 [id=e3e121c36ff44ea0a3a98185c8cce381]) successfully deleted
I20260808 13:35:39.758117 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 264ee45f8dfa4a05b0e5e812ce1b622c (table default.Table_43c8f783f487473eb3ea05192f470ef8 [id=b421a6c36f8e42ca96490c98ed957ccd]) successfully deleted
W20260808 13:35:39.759012 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 9423b380a62c467895e7fdff14756586 because the tablet was not found. No further retry: Not found: Tablet not found: 9423b380a62c467895e7fdff14756586
W20260808 13:35:39.759706 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 7e8cf06462484c99a860762a82f7ad64 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e8cf06462484c99a860762a82f7ad64 already in progress: creating tablet
I20260808 13:35:39.762701 6033 raft_consensus.cc:348] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.763454 6033 raft_consensus.cc:374] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:39.763711 6033 raft_consensus.cc:729] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:39.763577 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_9f3509b59d574bf8a9c2500fba9ff099"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:39.763434 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5668bb8655e949f494b5c2e988ae33d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:39.764408 6033 consensus_queue.cc:260] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:39.764950 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f3509b59d574bf8a9c2500fba9ff099 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:39.764967 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5668bb8655e949f494b5c2e988ae33d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:39.769711 6046 tablet_bootstrap.cc:654] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:39.780608 6026 tablet_bootstrap.cc:492] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:39.781286 6026 ts_tablet_manager.cc:1397] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.206s user 0.004s sys 0.014s
I20260808 13:35:39.781339 7199 tablet_replica.cc:330] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:39.782167 7199 raft_consensus.cc:2229] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:39.785864 7906 tablet_replica.cc:330] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:39.794364 6026 raft_consensus.cc:348] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.796447 7906 raft_consensus.cc:2229] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:39.797415 6026 raft_consensus.cc:374] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:39.799958 6026 raft_consensus.cc:729] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:39.800663 7906 raft_consensus.cc:2258] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:39.804493 6026 consensus_queue.cc:260] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.815092 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_aa07254565ac49d4aadd887511d6838b" } modify_external_catalogs: true
I20260808 13:35:39.815639 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_aa07254565ac49d4aadd887511d6838b" } modify_external_catalogs: true
W20260808 13:35:39.818572 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e because the tablet was not found. No further retry: Not found: Tablet not found: fc9b2e69fe27431dac9eb5ecb9e3e23e
W20260808 13:35:39.820984 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 56d8d5a3ca724141b58e4e2cea23be68 because the tablet was not found. No further retry: Not found: Tablet not found: 56d8d5a3ca724141b58e4e2cea23be68
I20260808 13:35:39.824649 7199 raft_consensus.cc:2258] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:39.826534 7199 ts_tablet_manager.cc:1905] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:39.832494 6046 tablet_bootstrap.cc:492] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:39.832654 6033 ts_tablet_manager.cc:1428] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.099s user 0.000s sys 0.006s
I20260808 13:35:39.832926 6046 ts_tablet_manager.cc:1397] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.241s user 0.009s sys 0.004s
I20260808 13:35:39.833853 6033 tablet_bootstrap.cc:492] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:39.835657 6026 ts_tablet_manager.cc:1428] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.054s user 0.007s sys 0.000s
I20260808 13:35:39.837826 6046 raft_consensus.cc:348] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.820997 7906 ts_tablet_manager.cc:1905] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:39.701256 7973 raft_consensus.cc:504] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.847877 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:39.838557 6046 raft_consensus.cc:374] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:39.850301 6046 raft_consensus.cc:729] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:39.858229 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:39.859437 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet b1847d0fecb843cdbc833ad674a503a4 (table default.Table_2d75fe6a74ad43789e515c67b5167ab3 [id=29db8794594e4221be8974258b5127e0]). Expected version 1 got 0
I20260808 13:35:39.858991 6046 consensus_queue.cc:260] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:39.861608 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_5a0fd24800424aa7b4fa9120704dea8d [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 2 got 0
I20260808 13:35:39.863090 6046 ts_tablet_manager.cc:1428] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.030s user 0.006s sys 0.002s
I20260808 13:35:39.868606 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:39.870059 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:39.870875 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:39.871433 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 2bb8c7eeec164485a4e5bb80f9d5fd72 (table default.Table_62ad61e62a5b49b480ec08bc990efa85 [id=8e1d239d7f7446fa89184bf11ddb88d9]). Expected version 1 got 0
I20260808 13:35:39.871390 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adf9a03742a14cfe867c578c466838de" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:39.872148 5640 raft_consensus.cc:3037] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:39.868968 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adf9a03742a14cfe867c578c466838de" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:39.879477 5771 leader_election.cc:341] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:39.880090 6046 tablet_bootstrap.cc:492] T 72c4ddec62c242c88b3a759d8361cd30 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:39.901607 7013 ts_tablet_manager.cc:1918] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:39.902192 7013 log.cc:1196] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/9423b380a62c467895e7fdff14756586
I20260808 13:35:39.908268 6046 tablet_bootstrap.cc:654] T 72c4ddec62c242c88b3a759d8361cd30 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:39.908859 7013 ts_tablet_manager.cc:1939] T 9423b380a62c467895e7fdff14756586 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:39.931556 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 376) took 3717 ms. Trace:
I20260808 13:35:39.931950 7013 rpcz_store.cc:276] 0808 13:35:36.214045 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.214197 (+ 152us) service_pool.cc:227] Handling call
0808 13:35:39.723874 (+3509677us) ts_tablet_manager.cc:1145] Deleting tablet 9423b380a62c467895e7fdff14756586
0808 13:35:39.723913 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:39.841956 (+118043us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.899365 (+ 57409us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.931495 (+ 32130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3508967}
I20260808 13:35:39.933492 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 377) took 3715 ms. Trace:
I20260808 13:35:39.933863 7013 rpcz_store.cc:276] 0808 13:35:36.218371 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.218503 (+ 132us) service_pool.cc:227] Handling call
0808 13:35:39.933186 (+3714683us) ts_tablet_manager.cc:1145] Deleting tablet e00f2e0b98af431d99894e8778257bc4
0808 13:35:39.933243 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:39.933421 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3713982}
I20260808 13:35:39.934538 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 9423b380a62c467895e7fdff14756586 (table default.Table_1fe1ec319d834d319cd9c35e801b1c91 [id=0f9219b97bfc4351ad5b584cfdec058f]) successfully deleted
I20260808 13:35:39.945225 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:39.952822 7013 tablet_replica.cc:330] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:39.954844 7013 raft_consensus.cc:2229] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:39.955399 7013 raft_consensus.cc:2258] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:39.958685 7013 ts_tablet_manager.cc:1905] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:39.959205 4841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f
I20260808 13:35:39.969790 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:39.972352 4836 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_13ca344305b74220b866f3194259e283" } new_table_name: "default.Table_a14f7675417240ca88f88da86055bceb" modify_external_catalogs: true
I20260808 13:35:39.972579 4840 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_323ff643b74d4f8e8a82e3fe8d1fc10c" } new_table_name: "default.Table_20c0fcbe614b48089d98badce0ef9eb2" modify_external_catalogs: true
I20260808 13:35:39.973940 7906 ts_tablet_manager.cc:1918] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:39.974345 7906 log.cc:1196] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/8b57966a9b7b4673ab9f98720cfd456d
I20260808 13:35:39.974475 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:39.974172 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_1955ad58908247a3b3002dc6a8ff2336"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:39.975801 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1955ad58908247a3b3002dc6a8ff2336 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:39.976368 5640 raft_consensus.cc:2454] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:39.977487 5772 leader_election.cc:302] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:39.980080 5826 tablet_service.cc:1514] Processing DeleteTablet for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:39.982080 7915 raft_consensus.cc:2781] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:39.982287 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:39.982407 6026 tablet_bootstrap.cc:492] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:39.982717 7915 raft_consensus.cc:686] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:39.981638 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_0e051b613c6e452f89c46cb5aff7f7dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:39.983965 7906 ts_tablet_manager.cc:1939] T 8b57966a9b7b4673ab9f98720cfd456d P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
W20260808 13:35:39.984052 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e051b613c6e452f89c46cb5aff7f7dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:39.983623 7915 consensus_queue.cc:237] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:39.990800 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet e00f2e0b98af431d99894e8778257bc4 because the tablet was not found. No further retry: Not found: Tablet not found: e00f2e0b98af431d99894e8778257bc4
I20260808 13:35:39.991432 6026 tablet_bootstrap.cc:654] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:40.007390 6046 tablet_bootstrap.cc:492] T 72c4ddec62c242c88b3a759d8361cd30 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:40.007953 6046 ts_tablet_manager.cc:1397] T 72c4ddec62c242c88b3a759d8361cd30 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.128s user 0.016s sys 0.004s
I20260808 13:35:40.010160 6046 raft_consensus.cc:348] T 72c4ddec62c242c88b3a759d8361cd30 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.016398 4838 catalog_manager.cc:5495] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:40.017871 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:40.018678 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:40.019520 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:40.020504 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:40.021486 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:40.026135 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet b1847d0fecb843cdbc833ad674a503a4 (table default.Table_2d75fe6a74ad43789e515c67b5167ab3 [id=29db8794594e4221be8974258b5127e0]). Expected version 1 got 0
I20260808 13:35:40.024544 5338 tablet_service.cc:1467] Processing CreateTablet for tablet 48e8293aa32144bcb89047096c88ffec (DEFAULT_TABLE table=default.Table_d1a5d56bb11c4d0a87b86c1b82f4180f [id=99cf3d4080734790a0f96ac8b1239008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.027602 5338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e8293aa32144bcb89047096c88ffec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.028082 4838 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]). Expected version 1 got 0
I20260808 13:35:40.029182 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 0b316390787f4170956f1c47a525d430
I20260808 13:35:40.033342 4832 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013] alter complete (version 6)
I20260808 13:35:40.033921 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:40.035451 7961 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:40.037252 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 677154c6f4584ee0bb103ab6f27fa9ea (DEFAULT_TABLE table=default.Table_465ac7f2a60c415a8809fdfccb4b5e27 [id=f5be929e60354ded904b228314f820d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.038597 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 677154c6f4584ee0bb103ab6f27fa9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.038898 6046 raft_consensus.cc:374] T 72c4ddec62c242c88b3a759d8361cd30 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:40.039171 6046 raft_consensus.cc:729] T 72c4ddec62c242c88b3a759d8361cd30 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:40.039865 6046 consensus_queue.cc:260] T 72c4ddec62c242c88b3a759d8361cd30 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.045457 6046 ts_tablet_manager.cc:1428] T 72c4ddec62c242c88b3a759d8361cd30 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.037s user 0.005s sys 0.000s
I20260808 13:35:40.046288 6046 tablet_bootstrap.cc:492] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:39.979305 7973 leader_election.cc:288] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:40.047104 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b316390787f4170956f1c47a525d430 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:40.050055 5826 tablet_service.cc:1467] Processing CreateTablet for tablet 48e8293aa32144bcb89047096c88ffec (DEFAULT_TABLE table=default.Table_d1a5d56bb11c4d0a87b86c1b82f4180f [id=99cf3d4080734790a0f96ac8b1239008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.050880 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b316390787f4170956f1c47a525d430 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:40.052546 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 677154c6f4584ee0bb103ab6f27fa9ea (DEFAULT_TABLE table=default.Table_465ac7f2a60c415a8809fdfccb4b5e27 [id=f5be929e60354ded904b228314f820d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.053627 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 677154c6f4584ee0bb103ab6f27fa9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.057446 7973 raft_consensus.cc:482] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:40.057992 7973 raft_consensus.cc:504] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.061661 5826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e8293aa32144bcb89047096c88ffec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.063184 7013 ts_tablet_manager.cc:1918] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:40.063540 7013 log.cc:1196] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/56d8d5a3ca724141b58e4e2cea23be68
I20260808 13:35:40.064862 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b57966a9b7b4673ab9f98720cfd456d" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:40.065690 5381 raft_consensus.cc:2454] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:40.066694 6046 tablet_bootstrap.cc:654] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:40.067301 5771 leader_election.cc:302] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:40.068033 7560 raft_consensus.cc:2781] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:40.068343 7560 raft_consensus.cc:482] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:40.068615 7560 raft_consensus.cc:3037] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:40.078050 7013 ts_tablet_manager.cc:1939] T 56d8d5a3ca724141b58e4e2cea23be68 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:40.079155 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 379) took 3299 ms. Trace:
I20260808 13:35:40.079934 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b316390787f4170956f1c47a525d430 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:39 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:40.079490 7013 rpcz_store.cc:276] 0808 13:35:36.780018 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.780218 (+ 200us) service_pool.cc:227] Handling call
0808 13:35:39.952545 (+3172327us) ts_tablet_manager.cc:1145] Deleting tablet 56d8d5a3ca724141b58e4e2cea23be68
0808 13:35:39.952590 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.020124 (+ 67534us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.062964 (+ 42840us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.079101 (+ 16137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3171300}
I20260808 13:35:40.082351 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 56d8d5a3ca724141b58e4e2cea23be68 (table default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212]) successfully deleted
I20260808 13:35:40.082571 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 380) took 3270 ms. Trace:
I20260808 13:35:40.087167 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:40.090902 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 379) took 1086 ms. Trace:
I20260808 13:35:40.092531 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 677154c6f4584ee0bb103ab6f27fa9ea (DEFAULT_TABLE table=default.Table_465ac7f2a60c415a8809fdfccb4b5e27 [id=f5be929e60354ded904b228314f820d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.094436 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 48e8293aa32144bcb89047096c88ffec (DEFAULT_TABLE table=default.Table_d1a5d56bb11c4d0a87b86c1b82f4180f [id=99cf3d4080734790a0f96ac8b1239008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.096683 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d8d5a3ca724141b58e4e2cea23be68 with delete_type TABLET_DATA_DELETED (replaced by tablet 14a9373000e24e75858e6d7457587264) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:40.097335 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9bc338093f403881d329a7e913c9f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:20 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:40.100411 7560 raft_consensus.cc:504] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.101655 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 677154c6f4584ee0bb103ab6f27fa9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.103693 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e8293aa32144bcb89047096c88ffec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.104267 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b57966a9b7b4673ab9f98720cfd456d" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:40.105032 5381 raft_consensus.cc:3037] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:40.108765 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet 9423b380a62c467895e7fdff14756586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:40.113705 7199 ts_tablet_manager.cc:1918] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:40.114090 7199 log.cc:1196] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/264ee45f8dfa4a05b0e5e812ce1b622c
W20260808 13:35:40.118188 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 51) took 847 ms (client timeout 1000 ms). Trace:
W20260808 13:35:40.118559 4832 rpcz_store.cc:269] 0808 13:35:39.270448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.320135 (+ 49687us) service_pool.cc:227] Handling call
0808 13:35:39.891349 (+571214us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:39.891374 (+ 25us) write_op.cc:454] Released schema lock
0808 13:35:40.118128 (+226754us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:39.872669 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:39.873126 (+ 457us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:39.873159 (+ 33us) write_op.cc:435] Acquired schema lock
0808 13:35:39.873176 (+ 17us) tablet.cc:660] Decoding operations
0808 13:35:39.873490 (+ 314us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:39.873542 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:39.873573 (+ 31us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:39.873721 (+ 148us) tablet.cc:699] Row locks acquired
0808 13:35:39.873735 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:39.873874 (+ 139us) write_op.cc:270] Start()
0808 13:35:39.873956 (+ 82us) write_op.cc:276] Timestamp: P: 1786196139873797 usec, L: 0
0808 13:35:39.873972 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:39.874467 (+ 495us) log.cc:842] Serialized 356 byte log entry
0808 13:35:39.882804 (+ 8337us) op_driver.cc:444] REPLICATION: finished
0808 13:35:39.883210 (+ 406us) write_op.cc:301] APPLY: starting
0808 13:35:39.883269 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:39.883397 (+ 128us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:39.883542 (+ 145us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:39.883737 (+ 195us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:39.883980 (+ 243us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:39.884043 (+ 63us) write_op.cc:312] APPLY: finished
0808 13:35:39.884314 (+ 271us) log.cc:842] Serialized 23 byte log entry
0808 13:35:39.884542 (+ 228us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:39.884660 (+ 118us) write_op.cc:454] Released schema lock
0808 13:35:39.884709 (+ 49us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":236,"apply.run_cpu_time_us":1821,"apply.run_wall_time_us":1820,"num_ops":1,"prepare.queue_time_us":14748,"prepare.run_cpu_time_us":2091,"prepare.run_wall_time_us":2092,"raft.queue_time_us":1148,"raft.run_cpu_time_us":951,"raft.run_wall_time_us":952,"replication_time_us":8749}]]}
I20260808 13:35:40.091344 7906 rpcz_store.cc:276] 0808 13:35:39.004064 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.004940 (+ 876us) service_pool.cc:227] Handling call
0808 13:35:39.784793 (+779853us) ts_tablet_manager.cc:1145] Deleting tablet 8b57966a9b7b4673ab9f98720cfd456d
0808 13:35:39.785081 (+ 288us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:39.900455 (+115374us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:39.973637 (+ 73182us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.090821 (+117184us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1732,"tablet-delete.queue_time_us":779010}
I20260808 13:35:40.125211 7199 ts_tablet_manager.cc:1939] T 264ee45f8dfa4a05b0e5e812ce1b622c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:40.128484 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 16736500f94c43d3a3163f61b7606aad (DEFAULT_TABLE table=default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.126600 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 356) took 4542 ms. Trace:
I20260808 13:35:40.130173 7199 rpcz_store.cc:276] 0808 13:35:35.583894 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.584313 (+ 419us) service_pool.cc:227] Handling call
0808 13:35:39.745394 (+4161081us) ts_tablet_manager.cc:1145] Deleting tablet 264ee45f8dfa4a05b0e5e812ce1b622c
0808 13:35:39.781186 (+ 35792us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.006624 (+225438us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.113466 (+106842us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.126545 (+ 13079us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":41887,"tablet-delete.queue_time_us":4160505}
I20260808 13:35:40.131289 7906 tablet_replica.cc:330] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:40.132128 7906 raft_consensus.cc:2229] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:40.132464 6026 tablet_bootstrap.cc:492] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:40.132624 7906 raft_consensus.cc:2258] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:40.133066 6026 ts_tablet_manager.cc:1397] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.151s user 0.020s sys 0.002s
I20260808 13:35:40.133987 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 8bac8adbffa84d8e8f405305b92c4500 (DEFAULT_TABLE table=default.Table_41de7b9190a34c0f97e4c0d7fcc14a49 [id=bb50a39ab33849689bdaf347ed3e9fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.135862 6026 raft_consensus.cc:348] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.096995 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:16 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:40.137252 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 16736500f94c43d3a3163f61b7606aad (DEFAULT_TABLE table=default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.139312 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16736500f94c43d3a3163f61b7606aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.139894 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16736500f94c43d3a3163f61b7606aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.140352 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 357) took 4556 ms. Trace:
I20260808 13:35:40.143784 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bac8adbffa84d8e8f405305b92c4500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.144906 6026 raft_consensus.cc:374] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:40.145164 6026 raft_consensus.cc:729] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:40.140626 7199 rpcz_store.cc:276] 0808 13:35:35.584177 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:35.585013 (+ 836us) service_pool.cc:227] Handling call
0808 13:35:40.140113 (+4555100us) ts_tablet_manager.cc:1145] Deleting tablet 896389941179456d9d0ed3cd14726f11
0808 13:35:40.140151 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.140302 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4554568}
I20260808 13:35:40.146833 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 358) took 3990 ms. Trace:
I20260808 13:35:40.147449 6026 consensus_queue.cc:260] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.148213 7199 rpcz_store.cc:276] 0808 13:35:36.156752 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.156899 (+ 147us) service_pool.cc:227] Handling call
0808 13:35:40.146492 (+3989593us) ts_tablet_manager.cc:1145] Deleting tablet e5fb1d373c4d486ca3815023897721a2
0808 13:35:40.146531 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.146783 (+ 252us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3989024}
I20260808 13:35:40.154625 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 359) took 3343 ms. Trace:
I20260808 13:35:40.156572 7199 rpcz_store.cc:276] 0808 13:35:36.811395 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.811527 (+ 132us) service_pool.cc:227] Handling call
0808 13:35:40.154264 (+3342737us) ts_tablet_manager.cc:1145] Deleting tablet a11a2666f0b44d70a08a9195e8fcf69d
0808 13:35:40.154305 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.154553 (+ 248us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3341704}
I20260808 13:35:40.155906 6026 ts_tablet_manager.cc:1428] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260808 13:35:40.158188 7199 tablet_replica.cc:330] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:40.159148 7199 raft_consensus.cc:2229] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:40.169634 7199 raft_consensus.cc:2258] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:40.083225 7013 rpcz_store.cc:276] 0808 13:35:36.811962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.812114 (+ 152us) service_pool.cc:227] Handling call
0808 13:35:40.082192 (+3270078us) ts_tablet_manager.cc:1145] Deleting tablet a11a2666f0b44d70a08a9195e8fcf69d
0808 13:35:40.082228 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.082519 (+ 291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3269167}
I20260808 13:35:40.172935 7906 ts_tablet_manager.cc:1905] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:40.173638 7199 ts_tablet_manager.cc:1905] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:40.124399 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 8b57966a9b7b4673ab9f98720cfd456d (table default.Table_d70de767ab2548b4a9e45418eefc9a2b [id=10bb97e006904dc9b34ebedad3ec13de]) successfully deleted
I20260808 13:35:40.140678 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 264ee45f8dfa4a05b0e5e812ce1b622c (table default.Table_43c8f783f487473eb3ea05192f470ef8 [id=b421a6c36f8e42ca96490c98ed957ccd]) successfully deleted
I20260808 13:35:40.141057 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_b4732fab4b8b4c4f9d3490f76ecfe24c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:40.179020 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 896389941179456d9d0ed3cd14726f11 because the tablet was not found. No further retry: Not found: Tablet not found: 896389941179456d9d0ed3cd14726f11
I20260808 13:35:40.179205 6026 tablet_bootstrap.cc:492] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:40.179621 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet e5fb1d373c4d486ca3815023897721a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5fb1d373c4d486ca3815023897721a2 already in progress: creating tablet
W20260808 13:35:40.179319 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b4732fab4b8b4c4f9d3490f76ecfe24c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:40.180182 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet a11a2666f0b44d70a08a9195e8fcf69d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a11a2666f0b44d70a08a9195e8fcf69d already in progress: creating tablet
W20260808 13:35:40.180788 4821 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 56408.3us
I20260808 13:35:40.172968 7893 raft_consensus.cc:482] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:40.182770 4833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_0bde50fe531b4d5a8d76821fde24d7b1" } modify_external_catalogs: true
I20260808 13:35:40.182729 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 16736500f94c43d3a3163f61b7606aad (DEFAULT_TABLE table=default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.182732 7893 raft_consensus.cc:504] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.183981 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16736500f94c43d3a3163f61b7606aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.185519 7893 leader_election.cc:288] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:40.187906 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 8bac8adbffa84d8e8f405305b92c4500 (DEFAULT_TABLE table=default.Table_41de7b9190a34c0f97e4c0d7fcc14a49 [id=bb50a39ab33849689bdaf347ed3e9fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.188899 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bac8adbffa84d8e8f405305b92c4500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.143517 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_187b31571225409e855d0b4783e4b74c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:40.148586 4840 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c2c50fd06b8841c8b5a944401b495d2e consensus state has no leader
W20260808 13:35:40.171825 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet a11a2666f0b44d70a08a9195e8fcf69d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a11a2666f0b44d70a08a9195e8fcf69d already in progress: creating tablet
W20260808 13:35:40.195228 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_187b31571225409e855d0b4783e4b74c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:40.207125 6026 tablet_bootstrap.cc:654] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:40.217234 5829 tablet_service.cc:1467] Processing CreateTablet for tablet 8bac8adbffa84d8e8f405305b92c4500 (DEFAULT_TABLE table=default.Table_41de7b9190a34c0f97e4c0d7fcc14a49 [id=bb50a39ab33849689bdaf347ed3e9fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:40.218447 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bac8adbffa84d8e8f405305b92c4500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:40.222997 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196d80d806b0432cacf2ca6aa0da8c6b" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:40.227398 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196d80d806b0432cacf2ca6aa0da8c6b" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:40.228806 5550 leader_election.cc:341] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:40.229241 5551 leader_election.cc:341] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:40.229593 5551 leader_election.cc:302] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:40.230541 7893 raft_consensus.cc:2726] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:40.233078 6046 tablet_bootstrap.cc:492] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:40.238137 5381 raft_consensus.cc:2454] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:40.253046 6046 ts_tablet_manager.cc:1397] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.207s user 0.010s sys 0.002s
I20260808 13:35:40.269186 5771 leader_election.cc:302] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:40.279013 7957 raft_consensus.cc:2781] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:40.281832 6033 tablet_bootstrap.cc:654] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:40.279510 7957 raft_consensus.cc:686] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:40.284133 7973 leader_election.cc:288] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:40.309018 7957 consensus_queue.cc:237] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.312489 4832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_8d93597641274e5594c62600af306920" } modify_external_catalogs: true
I20260808 13:35:40.292547 7560 leader_election.cc:288] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:40.348048 4823 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0329c348c88a4ddda0f2b90be7a4af76 (table default.Table_a1db036771e0467fb083f890fefd54e9 [id=b449f7fe2512431790a1a39a4dde3570], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0329c348c88a4ddda0f2b90be7a4af76 consensus state has no leader
I20260808 13:35:40.356118 7013 tablet_replica.cc:330] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:40.357160 7013 raft_consensus.cc:2229] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:40.357676 7013 raft_consensus.cc:2258] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:40.360626 7013 ts_tablet_manager.cc:1905] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:40.368323 7893 raft_consensus.cc:482] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:40.368769 7893 raft_consensus.cc:504] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.355255 6046 raft_consensus.cc:348] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.381486 6046 raft_consensus.cc:374] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:40.381757 6046 raft_consensus.cc:729] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:35:40.405483 5772 leader_election.cc:341] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 8b57966a9b7b4673ab9f98720cfd456d
I20260808 13:35:40.406742 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:40.407946 7013 ts_tablet_manager.cc:1918] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260808 13:35:40.408006 5551 leader_election.cc:341] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:40.371941 7893 leader_election.cc:288] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:40.413195 5772 leader_election.cc:341] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 8b57966a9b7b4673ab9f98720cfd456d
I20260808 13:35:40.418802 7013 log.cc:1196] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/328d361c336848689c6906ca6fe2ccc3
I20260808 13:35:40.419548 7013 ts_tablet_manager.cc:1939] T 328d361c336848689c6906ca6fe2ccc3 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:40.420318 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 393) took 2079 ms. Trace:
I20260808 13:35:40.420609 7013 rpcz_store.cc:276] 0808 13:35:38.340739 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.340855 (+ 116us) service_pool.cc:227] Handling call
0808 13:35:40.171543 (+1830688us) ts_tablet_manager.cc:1145] Deleting tablet 328d361c336848689c6906ca6fe2ccc3
0808 13:35:40.355943 (+184400us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.373390 (+ 17447us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.407743 (+ 34353us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.420258 (+ 12515us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1830110}
I20260808 13:35:40.421705 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 328d361c336848689c6906ca6fe2ccc3 (table default.Table_b23ea067bd8149539074d42c1596a2aa [id=3a77dc355690462ba633f54fadbeb544]) successfully deleted
I20260808 13:35:40.382748 6046 consensus_queue.cc:260] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.422950 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 398) took 2047 ms. Trace:
I20260808 13:35:40.415531 5381 raft_consensus.cc:3037] T 72c4ddec62c242c88b3a759d8361cd30 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:40.423296 7013 rpcz_store.cc:276] 0808 13:35:38.375360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.377913 (+ 2553us) service_pool.cc:227] Handling call
0808 13:35:40.422618 (+2044705us) ts_tablet_manager.cc:1145] Deleting tablet 7e8cf06462484c99a860762a82f7ad64
0808 13:35:40.422656 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.422903 (+ 247us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2019652}
I20260808 13:35:40.424168 6046 ts_tablet_manager.cc:1428] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.170s user 0.009s sys 0.001s
W20260808 13:35:40.424484 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 7e8cf06462484c99a860762a82f7ad64 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e8cf06462484c99a860762a82f7ad64 already in progress: creating tablet
I20260808 13:35:40.424989 6046 tablet_bootstrap.cc:492] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:40.425434 7013 tablet_replica.cc:330] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:40.426214 7013 raft_consensus.cc:2229] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:40.426736 7013 raft_consensus.cc:2258] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:40.428618 7013 ts_tablet_manager.cc:1905] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:40.439288 4838 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_62ad61e62a5b49b480ec08bc990efa85 [id=8e1d239d7f7446fa89184bf11ddb88d9] alter complete (version 1)
I20260808 13:35:40.391810 7690 raft_consensus.cc:482] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:40.440094 7690 raft_consensus.cc:504] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.442176 7690 leader_election.cc:288] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:40.447960 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "275b6aeaad044ea1a16ecd82f63dc59f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:40.448627 5384 raft_consensus.cc:2454] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:40.449949 5551 leader_election.cc:302] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:40.450834 7893 raft_consensus.cc:2781] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:40.451162 7893 raft_consensus.cc:482] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:40.451468 7893 raft_consensus.cc:3037] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:40.456728 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 264ee45f8dfa4a05b0e5e812ce1b622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:40.461298 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 69642fdba1f145438b04eae9f8154101 with delete_type TABLET_DATA_DELETED (replaced by tablet 538d9a883e9d46668fde30876352d20e) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:40.474102 6046 tablet_bootstrap.cc:654] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:40.481578 7906 ts_tablet_manager.cc:1918] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:40.482110 7906 log.cc:1196] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/369a942248bd4cec9ee9170fe55601a3
I20260808 13:35:40.484768 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6b4561dcd8e64216a27618e3599c13d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:40.486310 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b4561dcd8e64216a27618e3599c13d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:40.466521 7893 raft_consensus.cc:504] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.512234 7199 ts_tablet_manager.cc:1918] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:40.514943 6026 tablet_bootstrap.cc:492] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:40.515359 6026 ts_tablet_manager.cc:1397] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.336s user 0.014s sys 0.001s
I20260808 13:35:40.518296 6026 raft_consensus.cc:348] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.518993 8032 raft_consensus.cc:482] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:40.519220 6026 raft_consensus.cc:374] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:40.519629 6026 raft_consensus.cc:729] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:40.520375 6026 consensus_queue.cc:260] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.521010 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "275b6aeaad044ea1a16ecd82f63dc59f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:40.521552 5384 raft_consensus.cc:3037] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:40.522277 6026 ts_tablet_manager.cc:1428] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 13:35:40.527031 7893 leader_election.cc:288] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:40.528705 6026 tablet_bootstrap.cc:492] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:40.541661 6026 tablet_bootstrap.cc:654] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:40.519447 8032 raft_consensus.cc:504] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.571369 8032 leader_election.cc:288] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:40.572563 7906 ts_tablet_manager.cc:1939] T 369a942248bd4cec9ee9170fe55601a3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:40.585445 7893 consensus_queue.cc:1035] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:40.604115 6026 tablet_bootstrap.cc:492] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:40.604559 6026 ts_tablet_manager.cc:1397] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.076s user 0.016s sys 0.003s
I20260808 13:35:40.550199 5384 raft_consensus.cc:2454] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:40.555577 7013 ts_tablet_manager.cc:1918] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:40.614871 7013 log.cc:1196] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/11cd8f12cda94c4eb9a848f002a4020b
I20260808 13:35:40.615694 7013 ts_tablet_manager.cc:1939] T 11cd8f12cda94c4eb9a848f002a4020b P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:40.615571 5551 leader_election.cc:302] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:40.616554 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 403) took 1567 ms. Trace:
I20260808 13:35:40.616878 7013 rpcz_store.cc:276] 0808 13:35:39.049506 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.049627 (+ 121us) service_pool.cc:227] Handling call
0808 13:35:40.425290 (+1375663us) ts_tablet_manager.cc:1145] Deleting tablet 11cd8f12cda94c4eb9a848f002a4020b
0808 13:35:40.425331 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.482944 (+ 57613us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.555339 (+ 72395us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.616504 (+ 61165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1375088}
I20260808 13:35:40.570101 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8cf06462484c99a860762a82f7ad64" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:40.600715 5462 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:35:40.491084 4833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_0bde50fe531b4d5a8d76821fde24d7b1" } modify_external_catalogs: true
I20260808 13:35:40.620249 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 11cd8f12cda94c4eb9a848f002a4020b (table default.Table_2b5f163ace454f5e87d04751846435e8 [id=d5b2a1e2eebe4f1695a98f926a8cb628]) successfully deleted
I20260808 13:35:40.618134 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 404) took 1524 ms. Trace:
I20260808 13:35:40.618844 7881 raft_consensus.cc:482] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:40.621403 7013 rpcz_store.cc:276] 0808 13:35:39.093342 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.093491 (+ 149us) service_pool.cc:227] Handling call
0808 13:35:40.617913 (+1524422us) ts_tablet_manager.cc:1145] Deleting tablet 896389941179456d9d0ed3cd14726f11
0808 13:35:40.617949 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.618092 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1523808}
I20260808 13:35:40.621729 7881 raft_consensus.cc:504] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.505457 4835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 107309c7ada246b6a15a784d7a8c399d
W20260808 13:35:40.622786 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 896389941179456d9d0ed3cd14726f11 because the tablet was not found. No further retry: Not found: Tablet not found: 896389941179456d9d0ed3cd14726f11
I20260808 13:35:40.623978 4835 catalog_manager.cc:6565] ReplaceTablet: tablet 107309c7ada246b6a15a784d7a8c399d deleted and replaced by tablet 5ae01fdfbc564fb8abf9b6bf9a308837
I20260808 13:35:40.624310 4835 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60270 (request call id 426) took 2143 ms. Trace:
I20260808 13:35:40.623955 6026 raft_consensus.cc:348] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.624616 6026 raft_consensus.cc:374] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:40.624891 6026 raft_consensus.cc:729] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:40.624969 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 405) took 1529 ms. Trace:
I20260808 13:35:40.624985 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b07b7d31ef41e484389c0f007598f0" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:40.625519 5640 raft_consensus.cc:2454] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:40.625240 7013 rpcz_store.cc:276] 0808 13:35:39.095623 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.095756 (+ 133us) service_pool.cc:227] Handling call
0808 13:35:40.624667 (+1528911us) ts_tablet_manager.cc:1145] Deleting tablet 2cd6945ecccc4684a3fa5b2a1f07057f
0808 13:35:40.624707 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.624921 (+ 214us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1528182}
I20260808 13:35:40.625456 6026 consensus_queue.cc:260] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:40.626365 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 2cd6945ecccc4684a3fa5b2a1f07057f because the tablet was not found. No further retry: Not found: Tablet not found: 2cd6945ecccc4684a3fa5b2a1f07057f
I20260808 13:35:40.533649 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:40.627550 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 406) took 1530 ms. Trace:
I20260808 13:35:40.627925 7013 rpcz_store.cc:276] 0808 13:35:39.097470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.097580 (+ 110us) service_pool.cc:227] Handling call
0808 13:35:40.627141 (+1529561us) ts_tablet_manager.cc:1145] Deleting tablet 69642fdba1f145438b04eae9f8154101
0808 13:35:40.627206 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.627490 (+ 284us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1529025}
I20260808 13:35:40.628912 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet 107309c7ada246b6a15a784d7a8c399d with delete_type TABLET_DATA_DELETED (replaced by tablet 5ae01fdfbc564fb8abf9b6bf9a308837) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:40.629558 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 69642fdba1f145438b04eae9f8154101 because the tablet was not found. No further retry: Not found: Tablet not found: 69642fdba1f145438b04eae9f8154101
I20260808 13:35:40.629711 5285 leader_election.cc:302] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:40.630573 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 408) took 1515 ms. Trace:
I20260808 13:35:40.630764 7881 leader_election.cc:288] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:40.631606 7881 raft_consensus.cc:2781] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260808 13:35:40.633224 5551 leader_election.cc:341] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:40.633896 7893 raft_consensus.cc:2781] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:40.636020 6026 ts_tablet_manager.cc:1428] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260808 13:35:40.636379 7881 raft_consensus.cc:482] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:40.635036 7893 raft_consensus.cc:686] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:40.631002 7013 rpcz_store.cc:276] 0808 13:35:39.114872 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.115109 (+ 237us) service_pool.cc:227] Handling call
0808 13:35:40.630341 (+1515232us) ts_tablet_manager.cc:1145] Deleting tablet d3682c9c656c4556bc13ae3aeddd7c5e
0808 13:35:40.630383 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.630526 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1514664}
W20260808 13:35:40.639801 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet d3682c9c656c4556bc13ae3aeddd7c5e because the tablet was not found. No further retry: Not found: Tablet not found: d3682c9c656c4556bc13ae3aeddd7c5e
I20260808 13:35:40.624560 4835 rpcz_store.cc:276] 0808 13:35:38.480866 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.204881 (+724015us) service_pool.cc:227] Handling call
0808 13:35:40.624254 (+1419373us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:40.144591 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:40.145905 (+ 1314us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:40.145928 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:40.145943 (+ 15us) tablet.cc:660] Decoding operations
0808 13:35:40.146251 (+ 308us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:40.146286 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:40.146309 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:40.146534 (+ 225us) tablet.cc:699] Row locks acquired
0808 13:35:40.146548 (+ 14us) write_op.cc:260] PREPARE: finished
0808 13:35:40.146897 (+ 349us) write_op.cc:270] Start()
0808 13:35:40.146988 (+ 91us) write_op.cc:276] Timestamp: P: 1786196140146846 usec, L: 0
0808 13:35:40.147016 (+ 28us) op_driver.cc:331] REPLICATION: starting
0808 13:35:40.161377 (+ 14361us) log.cc:842] Serialized 622 byte log entry
0808 13:35:40.161953 (+ 576us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:40.168554 (+ 6601us) op_driver.cc:444] REPLICATION: finished
0808 13:35:40.168850 (+ 296us) write_op.cc:301] APPLY: starting
0808 13:35:40.168915 (+ 65us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:40.169021 (+ 106us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:40.169040 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:40.169385 (+ 345us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:40.169664 (+ 279us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:40.169716 (+ 52us) write_op.cc:312] APPLY: finished
0808 13:35:40.169959 (+ 243us) log.cc:842] Serialized 29 byte log entry
0808 13:35:40.170123 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:40.170213 (+ 90us) write_op.cc:454] Released schema lock
0808 13:35:40.170258 (+ 45us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:40.170786 (+ 528us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:40.170826 (+ 40us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":149,"apply.run_cpu_time_us":1988,"apply.run_wall_time_us":2299,"num_ops":2,"prepare.queue_time_us":147424,"prepare.run_cpu_time_us":2748,"prepare.run_wall_time_us":17531,"raft.queue_time_us":652,"raft.run_cpu_time_us":584,"raft.run_wall_time_us":584,"replication_time_us":21456,"spinlock_wait_cycles":31773440}]]}
I20260808 13:35:40.639004 7881 raft_consensus.cc:3037] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:40.643859 7893 consensus_queue.cc:237] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.646433 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet 107309c7ada246b6a15a784d7a8c399d with delete_type TABLET_DATA_DELETED (replaced by tablet 5ae01fdfbc564fb8abf9b6bf9a308837) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:40.647519 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 409) took 1532 ms. Trace:
I20260808 13:35:40.649005 7013 rpcz_store.cc:276] 0808 13:35:39.115150 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.115247 (+ 97us) service_pool.cc:227] Handling call
0808 13:35:40.647218 (+1531971us) ts_tablet_manager.cc:1145] Deleting tablet e992a0294c404cfb9541b57782a05fc9
0808 13:35:40.647275 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.647465 (+ 190us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":53,"tablet-delete.queue_time_us":1531523}
I20260808 13:35:40.654717 4833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet d8b07b7d31ef41e484389c0f007598f0
W20260808 13:35:40.655344 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet e992a0294c404cfb9541b57782a05fc9 because the tablet was not found. No further retry: Not found: Tablet not found: e992a0294c404cfb9541b57782a05fc9
I20260808 13:35:40.657020 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b07b7d31ef41e484389c0f007598f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:40.581908 4838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 42) took 1090 ms (client timeout 1000 ms). Trace:
I20260808 13:35:40.505791 4832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_8d93597641274e5594c62600af306920" } modify_external_catalogs: true
I20260808 13:35:40.658849 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b07b7d31ef41e484389c0f007598f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:40.659303 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 410) took 1543 ms. Trace:
I20260808 13:35:40.660974 7013 rpcz_store.cc:276] 0808 13:35:39.116057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.116174 (+ 117us) service_pool.cc:227] Handling call
0808 13:35:40.658911 (+1542737us) ts_tablet_manager.cc:1145] Deleting tablet 127e8d0323954ef788018bcb91582951
0808 13:35:40.658965 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.659162 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1541997}
I20260808 13:35:40.669713 7013 tablet_replica.cc:330] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:40.670534 7013 raft_consensus.cc:2229] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:40.671298 7013 raft_consensus.cc:2258] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:40.676900 7013 ts_tablet_manager.cc:1905] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:40.693650 6026 tablet_bootstrap.cc:492] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:40.701599 7891 raft_consensus.cc:482] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:40.702229 7891 raft_consensus.cc:504] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.705490 7891 leader_election.cc:288] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:40.710418 6026 tablet_bootstrap.cc:654] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:40.726770 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 97119.4us
I20260808 13:35:40.732589 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:40.734820 7013 ts_tablet_manager.cc:1918] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:40.735479 7013 log.cc:1196] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/264ee45f8dfa4a05b0e5e812ce1b622c
I20260808 13:35:40.736703 7013 ts_tablet_manager.cc:1939] T 264ee45f8dfa4a05b0e5e812ce1b622c P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:40.739742 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "275b6aeaad044ea1a16ecd82f63dc59f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:40.728529 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 380) took 1711 ms. Trace:
I20260808 13:35:40.741324 7906 rpcz_store.cc:276] 0808 13:35:39.017594 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.018506 (+ 912us) service_pool.cc:227] Handling call
0808 13:35:40.123990 (+1105484us) ts_tablet_manager.cc:1145] Deleting tablet 369a942248bd4cec9ee9170fe55601a3
0808 13:35:40.131133 (+ 7143us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.411781 (+280648us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.481317 (+ 69536us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.728431 (+247114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1104735}
I20260808 13:35:40.748448 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b07b7d31ef41e484389c0f007598f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:40.748478 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 107309c7ada246b6a15a784d7a8c399d with delete_type TABLET_DATA_DELETED (replaced by tablet 5ae01fdfbc564fb8abf9b6bf9a308837) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:40.739796 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "851a0b374a854efd87dd66e39b9df2d5" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:40.749104 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 411) took 1619 ms. Trace:
W20260808 13:35:40.751351 5551 leader_election.cc:341] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:40.752481 5881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b07b7d31ef41e484389c0f007598f0" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:40.753803 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "851a0b374a854efd87dd66e39b9df2d5" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:40.760694 7906 tablet_replica.cc:330] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:40.749447 7013 rpcz_store.cc:276] 0808 13:35:39.130003 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.130138 (+ 135us) service_pool.cc:227] Handling call
0808 13:35:40.669542 (+1539404us) ts_tablet_manager.cc:1145] Deleting tablet 264ee45f8dfa4a05b0e5e812ce1b622c
0808 13:35:40.669590 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.707523 (+ 37933us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.714527 (+ 7004us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.749028 (+ 34501us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1538690}
I20260808 13:35:40.763310 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 412) took 1410 ms. Trace:
W20260808 13:35:40.763283 5550 leader_election.cc:341] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:40.763705 7906 raft_consensus.cc:2229] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:40.764559 7906 raft_consensus.cc:2258] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:40.769696 7906 ts_tablet_manager.cc:1905] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:40.770196 7013 rpcz_store.cc:276] 0808 13:35:39.352505 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.353635 (+ 1130us) service_pool.cc:227] Handling call
0808 13:35:40.762904 (+1409269us) ts_tablet_manager.cc:1145] Deleting tablet d0767140736242f18ef4689b112deefb
0808 13:35:40.762953 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.763248 (+ 295us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1408258}
I20260808 13:35:40.780261 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 413) took 1250 ms. Trace:
I20260808 13:35:40.780726 7013 rpcz_store.cc:276] 0808 13:35:39.530227 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.530361 (+ 134us) service_pool.cc:227] Handling call
0808 13:35:40.779762 (+1249401us) ts_tablet_manager.cc:1145] Deleting tablet 6e3ad9ca75b541798bc04d7701128faa
0808 13:35:40.779811 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.780172 (+ 361us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1226557}
I20260808 13:35:40.786726 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "275b6aeaad044ea1a16ecd82f63dc59f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:40.787789 7013 tablet_replica.cc:330] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:40.789096 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8cf06462484c99a860762a82f7ad64" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:40.796519 8055 tablet.cc:1722] T bcc0c6ab755c4a6887dabeeef9364201 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260808 13:35:40.812093 8057 tablet.cc:1722] T bcc0c6ab755c4a6887dabeeef9364201 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260808 13:35:40.816001 6033 tablet_bootstrap.cc:492] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:40.816493 6033 ts_tablet_manager.cc:1397] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.983s user 0.008s sys 0.013s
W20260808 13:35:40.829625 5771 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.560s user 0.011s sys 0.003s
W20260808 13:35:40.830011 5771 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.560s user 0.013s sys 0.004s
W20260808 13:35:40.832150 7957 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s user 0.006s sys 0.006s
W20260808 13:35:40.832646 7957 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s user 0.006s sys 0.006s
I20260808 13:35:40.512614 7199 log.cc:1196] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/56d8d5a3ca724141b58e4e2cea23be68
I20260808 13:35:40.854074 7199 ts_tablet_manager.cc:1939] T 56d8d5a3ca724141b58e4e2cea23be68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:40.763713 5550 leader_election.cc:302] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:40.862200 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 98964.2us
W20260808 13:35:40.864001 5550 leader_election.cc:341] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:40.886482 5283 leader_election.cc:341] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:40.881376 5772 consensus_peers.cc:487] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:40.889626 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 360) took 4069 ms. Trace:
I20260808 13:35:40.885089 7957 consensus_queue.cc:1035] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:40.875447 7891 raft_consensus.cc:2726] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:40.661687 4838 rpcz_store.cc:269] 0808 13:35:39.491869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.509271 (+ 17402us) service_pool.cc:227] Handling call
0808 13:35:39.622057 (+112786us) spinlock_profiling.cc:243] Waited 58.8 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:40.581833 (+959776us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:40.174723 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:40.175621 (+ 898us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:40.175642 (+ 21us) write_op.cc:435] Acquired schema lock
0808 13:35:40.175653 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:40.175896 (+ 243us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:40.175929 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:40.175950 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:40.176081 (+ 131us) tablet.cc:699] Row locks acquired
0808 13:35:40.176092 (+ 11us) write_op.cc:260] PREPARE: finished
0808 13:35:40.176182 (+ 90us) write_op.cc:270] Start()
0808 13:35:40.176282 (+ 100us) write_op.cc:276] Timestamp: P: 1786196140176137 usec, L: 0
0808 13:35:40.176301 (+ 19us) op_driver.cc:331] REPLICATION: starting
0808 13:35:40.176793 (+ 492us) log.cc:842] Serialized 527 byte log entry
0808 13:35:40.319442 (+142649us) op_driver.cc:444] REPLICATION: finished
0808 13:35:40.323152 (+ 3710us) write_op.cc:301] APPLY: starting
0808 13:35:40.323232 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:40.323378 (+ 146us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:40.323549 (+ 171us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:40.323784 (+ 235us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:40.323986 (+ 202us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:40.324050 (+ 64us) write_op.cc:312] APPLY: finished
0808 13:35:40.324315 (+ 265us) log.cc:842] Serialized 23 byte log entry
0808 13:35:40.324997 (+ 682us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:40.325094 (+ 97us) write_op.cc:454] Released schema lock
0808 13:35:40.325150 (+ 56us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:40.325442 (+ 292us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:40.325491 (+ 49us) write_op.cc:454] Released schema lock
Related trace 'op':
0808 13:35:40.364287 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:40.364740 (+ 453us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:40.364772 (+ 32us) write_op.cc:435] Acquired schema lock
0808 13:35:40.364791 (+ 19us) tablet.cc:660] Decoding operations
0808 13:35:40.365100 (+ 309us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:40.365151 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:40.365181 (+ 30us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:40.365363 (+ 182us) tablet.cc:699] Row locks acquired
0808 13:35:40.365381 (+ 18us) write_op.cc:260] PREPARE: finished
0808 13:35:40.365502 (+ 121us) write_op.cc:270] Start()
0808 13:35:40.365598 (+ 96us) write_op.cc:276] Timestamp: P: 1786196140365427 usec, L: 0
0808 13:35:40.365619 (+ 21us) op_driver.cc:331] REPLICATION: starting
0808 13:35:40.377892 (+ 12273us) log.cc:842] Serialized 356 byte log entry
0808 13:35:40.379927 (+ 2035us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:40.386435 (+ 6508us) op_driver.cc:444] REPLICATION: finished
0808 13:35:40.386704 (+ 269us) write_op.cc:301] APPLY: starting
0808 13:35:40.386758 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:40.386851 (+ 93us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:40.386868 (+ 17us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:40.387076 (+ 208us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:40.387199 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:40.387250 (+ 51us) write_op.cc:312] APPLY: finished
0808 13:35:40.387417 (+ 167us) log.cc:842] Serialized 23 byte log entry
0808 13:35:40.387971 (+ 554us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:40.388132 (+ 161us) write_op.cc:454] Released schema lock
0808 13:35:40.388163 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:40.388333 (+ 170us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:40.388363 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":28,"spinlock_wait_cycles":135299456,"child_traces":[["op",{"apply.queue_time_us":3328,"apply.run_cpu_time_us":1641,"apply.run_wall_time_us":2773,"num_ops":1,"prepare.queue_time_us":49784,"prepare.run_cpu_time_us":1802,"prepare.run_wall_time_us":2265,"replication_time_us":143083,"thread_start_us":57413,"threads_started":1}],["op",{"apply.queue_time_us":108,"apply.run_cpu_time_us":1260,"apply.run_wall_time_us":1797,"num_ops":1,"prepare.queue_time_us":81,"prepare.run_cpu_time_us":3187,"prepare.run_wall_time_us":15947,"raft.queue_time_us":319,"raft.run_cpu_time_us":554,"raft.run_wall_time_us":553,"replication_time_us":20740,"spinlock_wait_cycles":27063168}]]}
W20260808 13:35:40.668507 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 127e8d0323954ef788018bcb91582951 because the tablet was not found. No further retry: Not found: Tablet not found: 127e8d0323954ef788018bcb91582951
W20260808 13:35:40.901027 4820 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 232521us
I20260808 13:35:40.902344 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 264ee45f8dfa4a05b0e5e812ce1b622c (table default.Table_43c8f783f487473eb3ea05192f470ef8 [id=b421a6c36f8e42ca96490c98ed957ccd]) successfully deleted
W20260808 13:35:40.903059 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet d0767140736242f18ef4689b112deefb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0767140736242f18ef4689b112deefb already in progress: creating tablet
W20260808 13:35:40.903614 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 6e3ad9ca75b541798bc04d7701128faa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e3ad9ca75b541798bc04d7701128faa already in progress: creating tablet
W20260808 13:35:40.904196 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f already in progress: creating tablet
I20260808 13:35:40.686664 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_248e910ea5a34c55978753b74fe71832" } new_table_name: "default.Table_0a849c19427549aea0eabc1e0324725f" modify_external_catalogs: true
I20260808 13:35:40.907312 4832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet f0ba43d1479a4f9e9d08bdfa29e7157b
I20260808 13:35:40.694396 4835 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_1e7d0ae2ebb1437c83136cc4e91e8a23" } new_table_name: "default.Table_db29cf6251b947269c7a48cfc05ea2e5" modify_external_catalogs: true
I20260808 13:35:40.719242 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_f80c686d970c4c32966a483d401a1e52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:40.908548 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f80c686d970c4c32966a483d401a1e52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:40.909576 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet f0ba43d1479a4f9e9d08bdfa29e7157b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:40.720369 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5db9de8011f74b9f9192b39113dfa0c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:40.912102 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5db9de8011f74b9f9192b39113dfa0c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:40.914155 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 0b316390787f4170956f1c47a525d430 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b316390787f4170956f1c47a525d430 already in progress: creating tablet
I20260808 13:35:40.721707 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_432ac67683bd4fb4a2db65e5369a98c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:40.915120 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 56d8d5a3ca724141b58e4e2cea23be68 because the tablet was not found. No further retry: Not found: Tablet not found: 56d8d5a3ca724141b58e4e2cea23be68
W20260808 13:35:40.915452 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_432ac67683bd4fb4a2db65e5369a98c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:40.915781 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 1f9bc338093f403881d329a7e913c9f7 because the tablet was not found. No further retry: Not found: Tablet not found: 1f9bc338093f403881d329a7e913c9f7
W20260808 13:35:40.916460 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 9423b380a62c467895e7fdff14756586 because the tablet was not found. No further retry: Not found: Tablet not found: 9423b380a62c467895e7fdff14756586
W20260808 13:35:40.918329 5829 consensus_meta.cc:287] T 8bac8adbffa84d8e8f405305b92c4500 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.538s user 0.007s sys 0.002s
I20260808 13:35:40.758549 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 369a942248bd4cec9ee9170fe55601a3 (table default.Table_7d31aaee9dd04cabbbbca675bde3a2f4 [id=e9b13612cdc742eb88f83d7ce4e999fd]) successfully deleted
W20260808 13:35:40.921876 4821 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 163370us
I20260808 13:35:40.923427 6046 tablet_bootstrap.cc:492] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
W20260808 13:35:40.928579 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e because the tablet was not found. No further retry: Not found: Tablet not found: fc9b2e69fe27431dac9eb5ecb9e3e23e
I20260808 13:35:40.818658 6033 raft_consensus.cc:348] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.934973 6033 raft_consensus.cc:374] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:40.935431 6033 raft_consensus.cc:729] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:40.890004 7199 rpcz_store.cc:276] 0808 13:35:36.820039 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:36.820256 (+ 217us) service_pool.cc:227] Handling call
0808 13:35:40.158048 (+3337792us) ts_tablet_manager.cc:1145] Deleting tablet 56d8d5a3ca724141b58e4e2cea23be68
0808 13:35:40.158085 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.266958 (+108873us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.512005 (+245047us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.889571 (+377566us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3201,"tablet-delete.queue_time_us":3337094}
I20260808 13:35:40.939384 7199 tablet_replica.cc:330] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
W20260808 13:35:40.831951 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 562808us
I20260808 13:35:40.938485 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 56d8d5a3ca724141b58e4e2cea23be68 (table default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212]) successfully deleted
W20260808 13:35:40.943041 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 107309c7ada246b6a15a784d7a8c399d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 107309c7ada246b6a15a784d7a8c399d already in progress: creating tablet
I20260808 13:35:40.949110 7881 raft_consensus.cc:504] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:40.957501 4836 master_service.cc:946] ReplaceTablet: received request to replace tablet aa493b8652e149f988d41bee811f6a2f from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:40.673640 4842 catalog_manager.cc:5495] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:40.951148 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet f0ba43d1479a4f9e9d08bdfa29e7157b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:40.923815 6046 ts_tablet_manager.cc:1397] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.499s user 0.012s sys 0.000s
I20260808 13:35:40.980952 4842 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:40.981189 7013 raft_consensus.cc:2229] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:40.982697 7013 raft_consensus.cc:2258] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:40.985188 7199 raft_consensus.cc:2229] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:40.990164 7881 leader_election.cc:288] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:40.999155 7199 raft_consensus.cc:2258] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:41.009925 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 110112us
I20260808 13:35:41.029676 6026 tablet_bootstrap.cc:492] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:41.030177 6026 ts_tablet_manager.cc:1397] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.337s user 0.038s sys 0.033s
W20260808 13:35:41.035210 4833 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4561c9af06e14ed8a2ec4c7285e2ad99 consensus state has no leader
I20260808 13:35:40.991690 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b07b7d31ef41e484389c0f007598f0" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:41.036934 4835 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0d0018c5333b4fde8ad4ce874660eb97 consensus state has no leader
I20260808 13:35:40.936419 6033 consensus_queue.cc:260] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.038730 6033 ts_tablet_manager.cc:1428] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.222s user 0.006s sys 0.000s
W20260808 13:35:41.036659 5283 leader_election.cc:341] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:41.044386 6046 raft_consensus.cc:348] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.045079 6046 raft_consensus.cc:374] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:41.045317 6046 raft_consensus.cc:729] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:41.045832 6046 consensus_queue.cc:260] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.048069 7013 ts_tablet_manager.cc:1905] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:41.053481 4838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_291e70645d0a4f24b232391de2e62d9b" } modify_external_catalogs: true
I20260808 13:35:41.053997 4838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_291e70645d0a4f24b232391de2e62d9b" } modify_external_catalogs: true
I20260808 13:35:41.063927 6033 tablet_bootstrap.cc:492] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:41.085963 4839 master_service.cc:946] ReplaceTablet: received request to replace tablet 290081bbe4164acd8808041b76441f0d from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:41.113061 7199 ts_tablet_manager.cc:1905] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:41.068229 8070 tablet.cc:1722] T bcc0c6ab755c4a6887dabeeef9364201 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260808 13:35:41.133430 4836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet aa493b8652e149f988d41bee811f6a2f
I20260808 13:35:41.134403 4841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fb8b49faec0a429d95f595b94f65c8e1" } modify_external_catalogs: true
I20260808 13:35:41.134775 4836 catalog_manager.cc:6565] ReplaceTablet: tablet aa493b8652e149f988d41bee811f6a2f deleted and replaced by tablet 9ef8c23840c848ddb7d8b1023a839e94
I20260808 13:35:41.134946 4841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_fb8b49faec0a429d95f595b94f65c8e1" } modify_external_catalogs: true
I20260808 13:35:41.140048 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet aa493b8652e149f988d41bee811f6a2f with delete_type TABLET_DATA_DELETED (replaced by tablet 9ef8c23840c848ddb7d8b1023a839e94) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:41.140928 5829 tablet_service.cc:1514] Processing DeleteTablet for tablet aa493b8652e149f988d41bee811f6a2f with delete_type TABLET_DATA_DELETED (replaced by tablet 9ef8c23840c848ddb7d8b1023a839e94) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:41.143514 6026 raft_consensus.cc:348] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.144192 6026 raft_consensus.cc:374] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:41.144521 6026 raft_consensus.cc:729] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:41.145157 6026 consensus_queue.cc:260] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.146559 6026 ts_tablet_manager.cc:1428] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.116s user 0.004s sys 0.000s
I20260808 13:35:41.159759 6026 tablet_bootstrap.cc:492] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:41.160835 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_2d9183e53d7d4c5780eccdef66bb4f27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:41.162325 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d9183e53d7d4c5780eccdef66bb4f27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:41.164412 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 2483b3320e2e49db96d39254dd5755aa
I20260808 13:35:41.165629 6026 tablet_bootstrap.cc:654] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:41.167968 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet 2483b3320e2e49db96d39254dd5755aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:41.008397 7906 ts_tablet_manager.cc:1918] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:41.170759 5829 tablet_service.cc:1514] Processing DeleteTablet for tablet 2483b3320e2e49db96d39254dd5755aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:41.172336 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet aa493b8652e149f988d41bee811f6a2f with delete_type TABLET_DATA_DELETED (replaced by tablet 9ef8c23840c848ddb7d8b1023a839e94) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:41.173275 6046 ts_tablet_manager.cc:1428] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.193s user 0.004s sys 0.001s
I20260808 13:35:41.177435 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 2483b3320e2e49db96d39254dd5755aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:41.181494 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b07b7d31ef41e484389c0f007598f0" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:41.182080 5630 raft_consensus.cc:3037] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:41.168298 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet f0ba43d1479a4f9e9d08bdfa29e7157b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:41.184448 5948 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:35:41.172233 7906 log.cc:1196] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/11cd8f12cda94c4eb9a848f002a4020b
I20260808 13:35:41.016613 7893 consensus_queue.cc:1035] T 72c4ddec62c242c88b3a759d8361cd30 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.168s
W20260808 13:35:41.130424 5771 consensus_peers.cc:487] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11cd8f12cda94c4eb9a848f002a4020b. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:41.193771 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 63442.9us
W20260808 13:35:41.194726 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 310329us
I20260808 13:35:41.200853 7906 ts_tablet_manager.cc:1939] T 11cd8f12cda94c4eb9a848f002a4020b P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:41.202744 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 382) took 2159 ms. Trace:
I20260808 13:35:41.203156 7906 rpcz_store.cc:276] 0808 13:35:39.043088 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.046497 (+ 3409us) service_pool.cc:227] Handling call
0808 13:35:40.760443 (+1713946us) ts_tablet_manager.cc:1145] Deleting tablet 11cd8f12cda94c4eb9a848f002a4020b
0808 13:35:40.760520 (+ 77us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:40.873823 (+113303us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:40.900633 (+ 26810us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:41.202670 (+302037us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":547,"tablet-delete.queue_time_us":1713134}
I20260808 13:35:41.204600 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 11cd8f12cda94c4eb9a848f002a4020b (table default.Table_2b5f163ace454f5e87d04751846435e8 [id=d5b2a1e2eebe4f1695a98f926a8cb628]) successfully deleted
I20260808 13:35:41.205255 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 385) took 1850 ms. Trace:
I20260808 13:35:41.205642 7906 rpcz_store.cc:276] 0808 13:35:39.355216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.355349 (+ 133us) service_pool.cc:227] Handling call
0808 13:35:41.204771 (+1849422us) ts_tablet_manager.cc:1145] Deleting tablet d0767140736242f18ef4689b112deefb
0808 13:35:41.204832 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.205196 (+ 364us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1848922}
I20260808 13:35:41.206527 8032 raft_consensus.cc:482] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:41.206817 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet d0767140736242f18ef4689b112deefb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0767140736242f18ef4689b112deefb already in progress: creating tablet
I20260808 13:35:41.206974 8032 raft_consensus.cc:504] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.208850 8032 leader_election.cc:288] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:41.210597 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 387) took 1682 ms. Trace:
I20260808 13:35:41.214864 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a27517560d64a3b8da2cf55980e18a8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:41.217662 5772 consensus_peers.cc:487] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b57966a9b7b4673ab9f98720cfd456d. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:41.223889 7906 rpcz_store.cc:276] 0808 13:35:39.528528 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.528640 (+ 112us) service_pool.cc:227] Handling call
0808 13:35:41.206992 (+1678352us) ts_tablet_manager.cc:1145] Deleting tablet 6e3ad9ca75b541798bc04d7701128faa
0808 13:35:41.207036 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.210515 (+ 3479us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":23936,"tablet-delete.queue_time_us":1677751}
W20260808 13:35:41.225188 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 6e3ad9ca75b541798bc04d7701128faa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e3ad9ca75b541798bc04d7701128faa already in progress: creating tablet
I20260808 13:35:41.225925 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 388) took 1252 ms. Trace:
I20260808 13:35:41.226321 7906 rpcz_store.cc:276] 0808 13:35:39.973981 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.974191 (+ 210us) service_pool.cc:227] Handling call
0808 13:35:41.225425 (+1251234us) ts_tablet_manager.cc:1145] Deleting tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f
0808 13:35:41.225477 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.225854 (+ 377us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1250522}
I20260808 13:35:41.226923 6046 tablet_bootstrap.cc:492] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:41.227547 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f already in progress: creating tablet
W20260808 13:35:41.228508 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 217993us
W20260808 13:35:41.230768 5550 leader_election.cc:341] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:41.231181 5550 leader_election.cc:302] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:41.231884 7891 raft_consensus.cc:2726] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:41.234676 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 290081bbe4164acd8808041b76441f0d
W20260808 13:35:41.234635 5550 leader_election.cc:341] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:41.235554 5550 leader_election.cc:341] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:41.237324 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet 290081bbe4164acd8808041b76441f0d with delete_type TABLET_DATA_DELETED (replaced by tablet ccef7d478a31444481546a15a95e02d9) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:41.238951 4839 catalog_manager.cc:6565] ReplaceTablet: tablet 290081bbe4164acd8808041b76441f0d deleted and replaced by tablet ccef7d478a31444481546a15a95e02d9
I20260808 13:35:41.243938 4838 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_9f3509b59d574bf8a9c2500fba9ff099" } new_table_name: "default.Table_d20dd87e2f844e739e8f18ec0be12963" modify_external_catalogs: true
I20260808 13:35:41.245385 4840 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_d1a5d56bb11c4d0a87b86c1b82f4180f" } new_table_name: "default.Table_9b62f270f0aa43bf92edae3c1e5b0ffd" modify_external_catalogs: true
I20260808 13:35:41.248432 5829 tablet_service.cc:1514] Processing DeleteTablet for tablet 290081bbe4164acd8808041b76441f0d with delete_type TABLET_DATA_DELETED (replaced by tablet ccef7d478a31444481546a15a95e02d9) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:41.253377 5550 leader_election.cc:302] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:41.254196 7690 raft_consensus.cc:2726] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:41.264618 4841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet c8cc2a1b87b14d29838bbbe3de44d2a1
I20260808 13:35:41.268849 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet c8cc2a1b87b14d29838bbbe3de44d2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:41.269608 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a27517560d64a3b8da2cf55980e18a8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:41.273447 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet c8cc2a1b87b14d29838bbbe3de44d2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:41.276489 8048 raft_consensus.cc:482] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:41.276958 8048 raft_consensus.cc:504] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.279011 8048 leader_election.cc:288] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:41.280256 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196d80d806b0432cacf2ca6aa0da8c6b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:41.280903 5829 tablet_service.cc:1514] Processing DeleteTablet for tablet c8cc2a1b87b14d29838bbbe3de44d2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:41.281808 5283 leader_election.cc:341] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:41.283226 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196d80d806b0432cacf2ca6aa0da8c6b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:41.283878 5644 raft_consensus.cc:2454] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:41.284842 5285 leader_election.cc:302] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:41.285511 8048 raft_consensus.cc:2781] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:41.285832 8048 raft_consensus.cc:482] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:41.286139 8048 raft_consensus.cc:3037] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:41.220090 5551 leader_election.cc:341] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:41.216434 5630 raft_consensus.cc:2454] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:41.239074 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet 290081bbe4164acd8808041b76441f0d with delete_type TABLET_DATA_DELETED (replaced by tablet ccef7d478a31444481546a15a95e02d9) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:41.288473 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 68415.5us
I20260808 13:35:41.293970 5285 leader_election.cc:302] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:41.294582 5285 raft_consensus.cc:2670] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260808 13:35:41.295506 8048 raft_consensus.cc:504] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.298134 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196d80d806b0432cacf2ca6aa0da8c6b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:41.298771 5630 raft_consensus.cc:3037] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:41.299003 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196d80d806b0432cacf2ca6aa0da8c6b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:41.300231 5283 leader_election.cc:341] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:41.297407 8048 leader_election.cc:288] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:41.252822 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 389) took 1220 ms. Trace:
I20260808 13:35:41.305373 7906 rpcz_store.cc:276] 0808 13:35:40.032446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.079720 (+ 47274us) service_pool.cc:227] Handling call
0808 13:35:41.227618 (+1147898us) ts_tablet_manager.cc:1145] Deleting tablet 0b316390787f4170956f1c47a525d430
0808 13:35:41.227663 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.252750 (+ 25087us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1147230}
W20260808 13:35:41.306907 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 0b316390787f4170956f1c47a525d430 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b316390787f4170956f1c47a525d430 already in progress: creating tablet
I20260808 13:35:41.312953 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_0fdee57de12549eab898e92f30fe691d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:41.315896 4833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d99de9b6e1e64b48873233ac5849ea09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:41.317315 4833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d99de9b6e1e64b48873233ac5849ea09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:41.318509 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0fdee57de12549eab898e92f30fe691d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:41.328972 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_834da6ca85d64417964c7976b57c1329"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:41.330664 4838 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0553d86a78fe4cc2896efe960ba51a27 has no consensus state
I20260808 13:35:41.338047 6033 tablet_bootstrap.cc:654] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:41.346306 6026 tablet_bootstrap.cc:492] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:41.346823 6026 ts_tablet_manager.cc:1397] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.187s user 0.009s sys 0.003s
W20260808 13:35:41.332516 5550 leader_election.cc:341] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:41.351042 5550 leader_election.cc:302] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:41.352296 7690 raft_consensus.cc:2726] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:41.344877 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 393) took 1260 ms. Trace:
I20260808 13:35:41.360440 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:41.359372 7906 rpcz_store.cc:276] 0808 13:35:40.084744 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.087037 (+ 2293us) service_pool.cc:227] Handling call
0808 13:35:41.339575 (+1252538us) ts_tablet_manager.cc:1145] Deleting tablet 264ee45f8dfa4a05b0e5e812ce1b622c
0808 13:35:41.344604 (+ 5029us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.344803 (+ 199us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1245658}
I20260808 13:35:41.361495 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
W20260808 13:35:41.361816 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 264ee45f8dfa4a05b0e5e812ce1b622c because the tablet was not found. No further retry: Not found: Tablet not found: 264ee45f8dfa4a05b0e5e812ce1b622c
I20260808 13:35:41.362210 4832 catalog_manager.cc:5495] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:41.363431 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
W20260808 13:35:41.363636 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 107309c7ada246b6a15a784d7a8c399d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 107309c7ada246b6a15a784d7a8c399d already in progress: creating tablet
I20260808 13:35:41.364221 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:41.364498 7906 tablet_replica.cc:330] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:41.364984 4832 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
W20260808 13:35:41.334975 4840 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 48e8293aa32144bcb89047096c88ffec consensus state has no leader
I20260808 13:35:41.365574 7906 raft_consensus.cc:2229] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:41.366027 7906 raft_consensus.cc:2258] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:41.330829 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_834da6ca85d64417964c7976b57c1329 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:41.366894 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0b316390787f4170956f1c47a525d430 (table default.Table_f3144416bf1140aaaa78d9ec41319c6b [id=abda1a6df047407ca80dadf6d4ccb804], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0b316390787f4170956f1c47a525d430 consensus state has no leader
I20260808 13:35:41.367882 7906 ts_tablet_manager.cc:1905] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:41.368764 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_9fa5c49027684fe6a85a46895e121932" } modify_external_catalogs: true
I20260808 13:35:41.369330 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_9fa5c49027684fe6a85a46895e121932" } modify_external_catalogs: true
I20260808 13:35:41.374331 6026 raft_consensus.cc:348] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.375061 6026 raft_consensus.cc:374] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:41.375315 6026 raft_consensus.cc:729] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:41.375912 6026 consensus_queue.cc:260] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.338657 7915 raft_consensus.cc:482] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:41.381357 5630 raft_consensus.cc:2454] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:41.382234 6026 ts_tablet_manager.cc:1428] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260808 13:35:41.381795 7915 raft_consensus.cc:504] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.382596 5285 leader_election.cc:302] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:41.383121 6026 tablet_bootstrap.cc:492] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:41.384313 7915 leader_election.cc:288] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:41.385998 5772 leader_election.cc:341] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 369a942248bd4cec9ee9170fe55601a3
W20260808 13:35:41.356451 5771 consensus_peers.cc:487] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:41.401660 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 45325.6us
W20260808 13:35:41.403048 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 50925.3us
I20260808 13:35:41.406255 7013 ts_tablet_manager.cc:1918] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:41.406661 7013 log.cc:1196] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/d8b07b7d31ef41e484389c0f007598f0
I20260808 13:35:41.407289 7013 ts_tablet_manager.cc:1939] T d8b07b7d31ef41e484389c0f007598f0 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:41.408664 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet d8b07b7d31ef41e484389c0f007598f0 (table default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]) successfully deleted
I20260808 13:35:41.409241 7915 consensus_queue.cc:1035] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:41.423350 6026 tablet_bootstrap.cc:654] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:41.408345 7013 tablet_replica.cc:330] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:41.424539 8048 raft_consensus.cc:2781] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:41.447197 6033 tablet_bootstrap.cc:492] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:41.447671 6033 ts_tablet_manager.cc:1397] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.384s user 0.008s sys 0.003s
I20260808 13:35:41.444309 8048 raft_consensus.cc:686] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:41.449075 8048 consensus_queue.cc:237] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.453009 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet de6800eceb96475284fdbffb4d23e5e3
I20260808 13:35:41.456317 6046 tablet_bootstrap.cc:654] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:41.463913 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet de6800eceb96475284fdbffb4d23e5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:41.480182 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_2d9183e53d7d4c5780eccdef66bb4f27" } modify_external_catalogs: true
I20260808 13:35:41.480749 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_2d9183e53d7d4c5780eccdef66bb4f27" } modify_external_catalogs: true
I20260808 13:35:41.484512 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6932f06e7e1e4ec79c76ad6efea73902"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:41.485956 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6932f06e7e1e4ec79c76ad6efea73902 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:41.490536 8034 lock_manager.cc:586] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 5193375188705914492) tablet 00000000000000000000000000000000 cur holder 0x7b5800062400
I20260808 13:35:41.502170 4833 master_service.cc:946] ReplaceTablet: received request to replace tablet 3424ca753bcc4f448d314c56aba3323e from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:41.457685 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet de6800eceb96475284fdbffb4d23e5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:41.507658 7906 ts_tablet_manager.cc:1918] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:41.524690 7906 log.cc:1196] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/d8b07b7d31ef41e484389c0f007598f0
I20260808 13:35:41.525712 7906 ts_tablet_manager.cc:1939] T d8b07b7d31ef41e484389c0f007598f0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:41.541431 6046 tablet_bootstrap.cc:492] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
W20260808 13:35:41.541366 4784 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 55701.3us
I20260808 13:35:41.541874 6046 ts_tablet_manager.cc:1397] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.315s user 0.004s sys 0.005s
I20260808 13:35:41.544027 6026 tablet_bootstrap.cc:492] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:41.544605 6026 ts_tablet_manager.cc:1397] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.162s user 0.010s sys 0.000s
I20260808 13:35:41.549826 6046 raft_consensus.cc:348] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.550467 6046 raft_consensus.cc:374] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:41.551024 6046 raft_consensus.cc:729] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:41.551672 6046 consensus_queue.cc:260] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.553728 6046 ts_tablet_manager.cc:1428] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.012s user 0.004s sys 0.004s
I20260808 13:35:41.554631 6046 tablet_bootstrap.cc:492] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:41.556222 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet d8b07b7d31ef41e484389c0f007598f0 (table default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]) successfully deleted
I20260808 13:35:41.562810 7891 raft_consensus.cc:482] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:41.563517 7891 raft_consensus.cc:504] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.565263 6046 tablet_bootstrap.cc:654] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:41.569034 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa59a18ae624bc8975bee2c460895a7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:41.568722 6026 raft_consensus.cc:348] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:41.569525 6026 raft_consensus.cc:374] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:41.569823 6026 raft_consensus.cc:729] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
W20260808 13:35:41.570806 5551 leader_election.cc:341] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:41.573274 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet aa493b8652e149f988d41bee811f6a2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa493b8652e149f988d41bee811f6a2f already in progress: creating tablet
W20260808 13:35:41.574234 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 2483b3320e2e49db96d39254dd5755aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2483b3320e2e49db96d39254dd5755aa already in progress: creating tablet
W20260808 13:35:41.575217 5550 consensus_peers.cc:487] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:41.572340 7891 leader_election.cc:288] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:41.578480 6026 consensus_queue.cc:260] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:41.587014 6026 ts_tablet_manager.cc:1428] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.042s user 0.005s sys 0.003s
I20260808 13:35:41.587581 7906 tablet_replica.cc:330] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:41.588703 7906 raft_consensus.cc:2229] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:41.589335 7906 raft_consensus.cc:2258] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:41.591351 6026 tablet_bootstrap.cc:492] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:41.597553 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_c3cd224a54354169a0cba0ad45c3257e" } new_table_name: "default.Table_45fdb943ae1b499e98db51a5ee6589a2" modify_external_catalogs: true
I20260808 13:35:41.599984 4836 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_034b94572e274eff86802d9cae6c907d" } new_table_name: "default.Table_465b28acac534aa0960266bb6c7efd27" modify_external_catalogs: true
I20260808 13:35:41.607077 4840 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 67b5547a9efc4e5196dbc70ad0e6bf16
I20260808 13:35:41.606290 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_533328cbcc13486fb2b3c66451f55ac3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:41.620216 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_533328cbcc13486fb2b3c66451f55ac3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:41.627744 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_3e5cfb8683d745f08a7738ed4d2cd13a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:41.629822 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e5cfb8683d745f08a7738ed4d2cd13a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:41.662750 7906 ts_tablet_manager.cc:1905] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:41.668717 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_dfee9bf8bf9e4bab8982be9ddb880b00" } modify_external_catalogs: true
I20260808 13:35:41.669467 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_dfee9bf8bf9e4bab8982be9ddb880b00" } modify_external_catalogs: true
I20260808 13:35:41.686636 6046 tablet_bootstrap.cc:492] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:41.687135 6046 ts_tablet_manager.cc:1397] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.133s user 0.066s sys 0.045s
I20260808 13:35:41.688264 7013 raft_consensus.cc:2229] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:41.689154 7013 raft_consensus.cc:2258] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:41.689967 6046 raft_consensus.cc:348] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.458695 7199 ts_tablet_manager.cc:1918] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:41.458932 5829 tablet_service.cc:1514] Processing DeleteTablet for tablet de6800eceb96475284fdbffb4d23e5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:41.690819 7199 log.cc:1196] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/328d361c336848689c6906ca6fe2ccc3
I20260808 13:35:41.692018 7199 ts_tablet_manager.cc:1939] T 328d361c336848689c6906ca6fe2ccc3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
W20260808 13:35:41.698026 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 163573us
I20260808 13:35:41.698922 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa59a18ae624bc8975bee2c460895a7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:41.706284 6026 tablet_bootstrap.cc:654] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:41.718288 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 315800us
I20260808 13:35:41.736043 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 374) took 3824 ms. Trace:
I20260808 13:35:41.736438 7199 rpcz_store.cc:276] 0808 13:35:37.912075 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:37.912207 (+ 132us) service_pool.cc:227] Handling call
0808 13:35:40.939156 (+3026949us) ts_tablet_manager.cc:1145] Deleting tablet 328d361c336848689c6906ca6fe2ccc3
0808 13:35:40.939232 (+ 76us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.419207 (+479975us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:41.458448 (+ 39241us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:41.735965 (+277517us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4066,"tablet-delete.queue_time_us":3026361}
I20260808 13:35:41.737716 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 328d361c336848689c6906ca6fe2ccc3 (table default.Table_b23ea067bd8149539074d42c1596a2aa [id=3a77dc355690462ba633f54fadbeb544]) successfully deleted
W20260808 13:35:41.743716 4836 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7b128feff212456faa1769efd96f6bd8 consensus state has no leader
W20260808 13:35:41.746999 4833 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 37495b29698940edb2924eb612e94bd7 consensus state has no leader
I20260808 13:35:41.749015 4836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_0f52af74f5c34a0aa2f84a521c188673"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:41.755340 4836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f52af74f5c34a0aa2f84a521c188673 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:41.690591 6046 raft_consensus.cc:374] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:41.720112 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369a942248bd4cec9ee9170fe55601a3" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:41.693598 7013 ts_tablet_manager.cc:1905] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:41.777107 6046 raft_consensus.cc:729] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:41.781910 6046 consensus_queue.cc:260] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.792337 5381 raft_consensus.cc:2454] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
W20260808 13:35:41.792863 5550 leader_election.cc:341] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:41.793270 5550 leader_election.cc:302] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:41.794361 7891 raft_consensus.cc:2726] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:41.795091 5771 leader_election.cc:302] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:41.798062 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 379) took 3690 ms. Trace:
I20260808 13:35:41.798429 7199 rpcz_store.cc:276] 0808 13:35:38.107615 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.107732 (+ 117us) service_pool.cc:227] Handling call
0808 13:35:41.738711 (+3630979us) ts_tablet_manager.cc:1145] Deleting tablet 1f9bc338093f403881d329a7e913c9f7
0808 13:35:41.797768 (+ 59057us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.797991 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3592968}
W20260808 13:35:41.799630 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 1f9bc338093f403881d329a7e913c9f7 because the tablet was not found. No further retry: Not found: Tablet not found: 1f9bc338093f403881d329a7e913c9f7
I20260808 13:35:41.800853 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 378) took 3693 ms. Trace:
I20260808 13:35:41.801198 7199 rpcz_store.cc:276] 0808 13:35:38.107333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.108467 (+ 1134us) service_pool.cc:227] Handling call
0808 13:35:41.800339 (+3691872us) ts_tablet_manager.cc:1145] Deleting tablet 31fc9fdb5ef543da833f9af6ca5807a9
0808 13:35:41.800636 (+ 297us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.800802 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3654409}
W20260808 13:35:41.802348 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 31fc9fdb5ef543da833f9af6ca5807a9 because the tablet was not found. No further retry: Not found: Tablet not found: 31fc9fdb5ef543da833f9af6ca5807a9
I20260808 13:35:41.803366 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 380) took 3695 ms. Trace:
I20260808 13:35:41.803714 7199 rpcz_store.cc:276] 0808 13:35:38.107875 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.108672 (+ 797us) service_pool.cc:227] Handling call
0808 13:35:41.803112 (+3694440us) ts_tablet_manager.cc:1145] Deleting tablet 2cd6945ecccc4684a3fa5b2a1f07057f
0808 13:35:41.803152 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.803306 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3657038}
W20260808 13:35:41.804952 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 2cd6945ecccc4684a3fa5b2a1f07057f because the tablet was not found. No further retry: Not found: Tablet not found: 2cd6945ecccc4684a3fa5b2a1f07057f
I20260808 13:35:41.806047 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 382) took 3697 ms. Trace:
I20260808 13:35:41.806813 7957 raft_consensus.cc:2781] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:41.807183 7957 raft_consensus.cc:482] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:41.807518 7957 raft_consensus.cc:3037] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:41.806406 7199 rpcz_store.cc:276] 0808 13:35:38.108333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.110214 (+ 1881us) service_pool.cc:227] Handling call
0808 13:35:41.805720 (+3695506us) ts_tablet_manager.cc:1145] Deleting tablet 127e8d0323954ef788018bcb91582951
0808 13:35:41.805759 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.806000 (+ 241us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3656129}
I20260808 13:35:41.811626 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 383) took 3589 ms. Trace:
I20260808 13:35:41.811960 7199 rpcz_store.cc:276] 0808 13:35:38.222046 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:38.223438 (+ 1392us) service_pool.cc:227] Handling call
0808 13:35:41.811286 (+3587848us) ts_tablet_manager.cc:1145] Deleting tablet 7e8cf06462484c99a860762a82f7ad64
0808 13:35:41.811330 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.811578 (+ 248us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3566930}
I20260808 13:35:41.812912 7199 tablet_replica.cc:330] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:41.813735 7199 raft_consensus.cc:2229] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:41.818831 7957 raft_consensus.cc:504] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:41.827350 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 127e8d0323954ef788018bcb91582951 because the tablet was not found. No further retry: Not found: Tablet not found: 127e8d0323954ef788018bcb91582951
W20260808 13:35:41.828023 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 7e8cf06462484c99a860762a82f7ad64 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e8cf06462484c99a860762a82f7ad64 already in progress: creating tablet
I20260808 13:35:41.832652 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369a942248bd4cec9ee9170fe55601a3" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:41.833249 5381 raft_consensus.cc:3037] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:41.839986 7199 raft_consensus.cc:2258] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:41.841027 6026 tablet_bootstrap.cc:492] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:41.841583 6026 ts_tablet_manager.cc:1397] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.250s user 0.004s sys 0.018s
I20260808 13:35:41.842976 7199 ts_tablet_manager.cc:1905] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:41.844233 6026 raft_consensus.cc:348] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.844920 6026 raft_consensus.cc:374] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:41.845211 6026 raft_consensus.cc:729] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:41.845808 6026 consensus_queue.cc:260] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.849226 5381 raft_consensus.cc:2454] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:41.849720 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 809049a771224082a16c270d01ec4bc7
I20260808 13:35:41.850368 5771 leader_election.cc:302] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:41.851205 8106 raft_consensus.cc:2781] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:41.851656 8106 raft_consensus.cc:686] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:41.855373 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet 809049a771224082a16c270d01ec4bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:41.855702 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 809049a771224082a16c270d01ec4bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:41.856757 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet 809049a771224082a16c270d01ec4bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:41.856973 8106 consensus_queue.cc:237] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.829905 6033 raft_consensus.cc:348] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.865808 6033 raft_consensus.cc:374] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:41.866137 6033 raft_consensus.cc:729] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:41.873628 7891 consensus_queue.cc:1035] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:41.878847 7957 leader_election.cc:288] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:41.888854 5954 heartbeater.cc:347] Connected to a master server at 127.4.170.62:34069
I20260808 13:35:41.871074 6033 consensus_queue.cc:260] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:41.898026 6033 ts_tablet_manager.cc:1428] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.205s user 0.000s sys 0.005s
I20260808 13:35:41.899125 6033 tablet_bootstrap.cc:492] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:41.899313 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 64949.9us
I20260808 13:35:41.917383 6046 ts_tablet_manager.cc:1428] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.230s user 0.007s sys 0.001s
I20260808 13:35:41.929421 7906 ts_tablet_manager.cc:1918] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:41.929833 7906 log.cc:1196] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/f0ba43d1479a4f9e9d08bdfa29e7157b
I20260808 13:35:41.933943 6026 ts_tablet_manager.cc:1428] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.092s user 0.005s sys 0.000s
I20260808 13:35:41.941246 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_bf5259e68f5c4869bac79b5b0a6a0cd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:41.962882 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_256bb136799848f4891c3923692e5feb" } new_table_name: "default.Table_938ca078ec5e4411b4be33056272983c" modify_external_catalogs: true
I20260808 13:35:41.930512 7906 ts_tablet_manager.cc:1939] T f0ba43d1479a4f9e9d08bdfa29e7157b P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:41.967231 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5a0fd24800424aa7b4fa9120704dea8d" } new_table_name: "default.Table_a450e64d7a494bbcb8f35780fda0ccc8" modify_external_catalogs: true
I20260808 13:35:41.970075 6026 tablet_bootstrap.cc:492] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:41.973676 8128 tablet.cc:1722] T d0cc688386f3413ea939dda3a75d6541 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:41.976150 6026 tablet_bootstrap.cc:654] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:41.976231 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf5259e68f5c4869bac79b5b0a6a0cd0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:41.980472 4839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b4732fab4b8b4c4f9d3490f76ecfe24c" } modify_external_catalogs: true
I20260808 13:35:41.981041 4839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_b4732fab4b8b4c4f9d3490f76ecfe24c" } modify_external_catalogs: true
I20260808 13:35:41.958215 7013 ts_tablet_manager.cc:1918] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:41.983021 7013 log.cc:1196] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/f0ba43d1479a4f9e9d08bdfa29e7157b
I20260808 13:35:41.983784 7013 ts_tablet_manager.cc:1939] T f0ba43d1479a4f9e9d08bdfa29e7157b P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
W20260808 13:35:41.986855 8127 log.cc:924] Time spent T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.060s user 0.000s sys 0.001s
W20260808 13:35:41.994817 5772 leader_election.cc:341] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 369a942248bd4cec9ee9170fe55601a3
I20260808 13:35:41.995730 8130 tablet.cc:1722] T d0cc688386f3413ea939dda3a75d6541 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:41.998170 6046 tablet_bootstrap.cc:492] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:42.020438 7906 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 401) took 1009 ms. Trace:
I20260808 13:35:42.020812 7906 rpcz_store.cc:276] 0808 13:35:41.011084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.016420 (+ 5336us) service_pool.cc:227] Handling call
0808 13:35:41.583593 (+567173us) ts_tablet_manager.cc:1145] Deleting tablet f0ba43d1479a4f9e9d08bdfa29e7157b
0808 13:35:41.583672 (+ 79us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.834499 (+250827us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:41.929201 (+ 94702us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:42.020376 (+ 91175us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":399169}
I20260808 13:35:42.022122 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet f0ba43d1479a4f9e9d08bdfa29e7157b (table default.Table_8d93597641274e5594c62600af306920 [id=b5733467d1904db5b6df9045484d6a94]) successfully deleted
W20260808 13:35:42.023883 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet c8cc2a1b87b14d29838bbbe3de44d2a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8cc2a1b87b14d29838bbbe3de44d2a1 already in progress: creating tablet
I20260808 13:35:42.039095 6033 tablet_bootstrap.cc:654] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:42.049163 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 290081bbe4164acd8808041b76441f0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 290081bbe4164acd8808041b76441f0d already in progress: creating tablet
W20260808 13:35:42.051267 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet de6800eceb96475284fdbffb4d23e5e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de6800eceb96475284fdbffb4d23e5e3 already in progress: creating tablet
I20260808 13:35:42.093784 6033 tablet_bootstrap.cc:492] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:42.099547 6033 ts_tablet_manager.cc:1397] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.201s user 0.005s sys 0.008s
W20260808 13:35:42.115274 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d8e4af746774747a70d425068e8dd63 consensus state has no leader
I20260808 13:35:42.116587 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_e4f01562a52f4079b8b916b057cf8014"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:42.117602 6033 raft_consensus.cc:348] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:42.118243 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4f01562a52f4079b8b916b057cf8014 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:42.118389 6033 raft_consensus.cc:374] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:42.130967 6033 raft_consensus.cc:729] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:42.135504 6033 consensus_queue.cc:260] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.137410 4840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_f998dcf284c249d7a6db0e4b803a157a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:42.139086 4840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f998dcf284c249d7a6db0e4b803a157a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:42.154042 4839 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 802d77be985a48ab8dcc5977c3782363
I20260808 13:35:42.154989 4839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_faaabef2a0ab42648427c33cda679fb7" } modify_external_catalogs: true
I20260808 13:35:42.155476 4839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_faaabef2a0ab42648427c33cda679fb7" } modify_external_catalogs: true
I20260808 13:35:42.157306 6033 ts_tablet_manager.cc:1428] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.047s user 0.005s sys 0.002s
I20260808 13:35:42.167459 6033 tablet_bootstrap.cc:492] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:42.197544 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 809049a771224082a16c270d01ec4bc7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 809049a771224082a16c270d01ec4bc7 already in progress: creating tablet
I20260808 13:35:42.223462 6033 tablet_bootstrap.cc:654] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:42.234095 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_efea7e3ddc70498f94c05ddee5ba3d76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:42.242901 7690 raft_consensus.cc:482] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:42.244704 7690 raft_consensus.cc:504] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.255290 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5663fa6a1bb148c3afc9340477b05a4b" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:42.256776 5551 leader_election.cc:341] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:42.257557 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efea7e3ddc70498f94c05ddee5ba3d76 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:42.262315 7690 leader_election.cc:288] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:42.284358 5385 raft_consensus.cc:3037] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:42.287254 8124 consensus_queue.cc:1035] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:42.289940 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5663fa6a1bb148c3afc9340477b05a4b" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:42.292089 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 427) took 1341 ms. Trace:
I20260808 13:35:42.292436 7013 rpcz_store.cc:276] 0808 13:35:40.950854 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.950992 (+ 138us) service_pool.cc:227] Handling call
0808 13:35:41.408221 (+457229us) ts_tablet_manager.cc:1145] Deleting tablet f0ba43d1479a4f9e9d08bdfa29e7157b
0808 13:35:41.408255 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:41.785847 (+377592us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:41.957975 (+172128us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:42.292028 (+334053us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":251932,"tablet-delete.queue_time_us":428384}
I20260808 13:35:42.293946 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 428) took 1154 ms. Trace:
I20260808 13:35:42.293964 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet f0ba43d1479a4f9e9d08bdfa29e7157b (table default.Table_8d93597641274e5594c62600af306920 [id=b5733467d1904db5b6df9045484d6a94]) successfully deleted
I20260808 13:35:42.294229 7013 rpcz_store.cc:276] 0808 13:35:41.139764 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.139878 (+ 114us) service_pool.cc:227] Handling call
0808 13:35:42.293604 (+1153726us) ts_tablet_manager.cc:1145] Deleting tablet aa493b8652e149f988d41bee811f6a2f
0808 13:35:42.293645 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.293894 (+ 249us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1153148}
I20260808 13:35:42.295625 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 429) took 1128 ms. Trace:
I20260808 13:35:42.295969 7013 rpcz_store.cc:276] 0808 13:35:41.167696 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.167804 (+ 108us) service_pool.cc:227] Handling call
0808 13:35:42.295131 (+1127327us) ts_tablet_manager.cc:1145] Deleting tablet 2483b3320e2e49db96d39254dd5755aa
0808 13:35:42.295182 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.295569 (+ 387us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1126740}
I20260808 13:35:42.297252 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 430) took 1060 ms. Trace:
I20260808 13:35:42.297641 7013 rpcz_store.cc:276] 0808 13:35:41.237057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.237198 (+ 141us) service_pool.cc:227] Handling call
0808 13:35:42.296906 (+1059708us) ts_tablet_manager.cc:1145] Deleting tablet 290081bbe4164acd8808041b76441f0d
0808 13:35:42.296942 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.297186 (+ 244us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1059128}
W20260808 13:35:42.299901 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet aa493b8652e149f988d41bee811f6a2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa493b8652e149f988d41bee811f6a2f already in progress: creating tablet
W20260808 13:35:42.300683 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 2483b3320e2e49db96d39254dd5755aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2483b3320e2e49db96d39254dd5755aa already in progress: creating tablet
W20260808 13:35:42.301569 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 290081bbe4164acd8808041b76441f0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 290081bbe4164acd8808041b76441f0d already in progress: creating tablet
I20260808 13:35:42.302477 7013 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 431) took 1029 ms. Trace:
I20260808 13:35:42.304389 7013 rpcz_store.cc:276] 0808 13:35:41.273138 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.273265 (+ 127us) service_pool.cc:227] Handling call
0808 13:35:42.298529 (+1025264us) ts_tablet_manager.cc:1145] Deleting tablet c8cc2a1b87b14d29838bbbe3de44d2a1
0808 13:35:42.298576 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.302431 (+ 3855us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1024700}
W20260808 13:35:42.305923 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet c8cc2a1b87b14d29838bbbe3de44d2a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8cc2a1b87b14d29838bbbe3de44d2a1 already in progress: creating tablet
W20260808 13:35:42.309348 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet de6800eceb96475284fdbffb4d23e5e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de6800eceb96475284fdbffb4d23e5e3 already in progress: creating tablet
W20260808 13:35:42.313486 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 809049a771224082a16c270d01ec4bc7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 809049a771224082a16c270d01ec4bc7 already in progress: creating tablet
W20260808 13:35:42.314226 5550 leader_election.cc:341] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:42.314757 5550 leader_election.cc:302] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:42.316879 8073 raft_consensus.cc:2726] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:42.350850 6046 tablet_bootstrap.cc:654] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:42.353585 4840 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d8357611ced42db91bd1e416f0e26b3 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:42.354916 7690 raft_consensus.cc:482] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:42.366946 7690 raft_consensus.cc:504] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:42.369031 7690 leader_election.cc:288] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:42.355217 8139 tablet.cc:1722] T d0cc688386f3413ea939dda3a75d6541 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:42.382088 6026 tablet_bootstrap.cc:492] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:42.382589 6026 ts_tablet_manager.cc:1397] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.413s user 0.019s sys 0.044s
W20260808 13:35:42.389433 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 37495b29698940edb2924eb612e94bd7 (table default.Table_45fdb943ae1b499e98db51a5ee6589a2 [id=ed0521e8ecae4c609b2a7426e5a565c0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 37495b29698940edb2924eb612e94bd7 consensus state has no leader
I20260808 13:35:42.397328 6026 raft_consensus.cc:348] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.411170 6026 raft_consensus.cc:374] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:42.411600 6026 raft_consensus.cc:729] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:42.419792 5720 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:35:42.412444 6026 consensus_queue.cc:260] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.437698 6026 ts_tablet_manager.cc:1428] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.055s user 0.005s sys 0.001s
I20260808 13:35:42.458633 8141 raft_consensus.cc:482] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:42.459273 8141 raft_consensus.cc:504] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.471784 8141 leader_election.cc:288] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:42.472311 6026 tablet_bootstrap.cc:492] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:42.477892 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362dbd6990e740b39690537af98aa2f8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:42.479772 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85b4fd30d6c4903946891a399acf7a0" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:42.489822 5551 leader_election.cc:341] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:42.491206 8034 lock_manager.cc:586] Waited 2 seconds to obtain row lock on key '<redacted>' (key hash 5193375188705914492) tablet 00000000000000000000000000000000 cur holder 0x7b5800062400
W20260808 13:35:42.497440 5551 leader_election.cc:341] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:42.516327 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_5ede676dcd5843a3b896699541bf6d01" } new_table_name: "default.Table_4cb4f6347fda4d268fd87065c9b063ea" modify_external_catalogs: true
I20260808 13:35:42.538472 6026 tablet_bootstrap.cc:654] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:42.561144 6026 tablet_bootstrap.cc:492] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:42.563876 6026 ts_tablet_manager.cc:1397] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.125s user 0.011s sys 0.000s
I20260808 13:35:42.571652 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362dbd6990e740b39690537af98aa2f8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:42.566866 6026 raft_consensus.cc:348] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.587227 6026 raft_consensus.cc:374] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:42.587651 6026 raft_consensus.cc:729] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:42.599138 6026 consensus_queue.cc:260] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.609175 6026 ts_tablet_manager.cc:1428] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.045s user 0.006s sys 0.000s
I20260808 13:35:42.610230 6026 tablet_bootstrap.cc:492] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:42.627300 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85b4fd30d6c4903946891a399acf7a0" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:42.628283 5877 raft_consensus.cc:2454] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:42.644778 6026 tablet_bootstrap.cc:654] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:42.649525 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 109997us
W20260808 13:35:42.662189 5550 leader_election.cc:341] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:42.662758 5550 leader_election.cc:302] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:42.664053 5550 leader_election.cc:302] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:42.664808 7690 raft_consensus.cc:2726] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:42.665133 8141 raft_consensus.cc:2781] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:42.665499 8141 raft_consensus.cc:482] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:42.665879 8141 raft_consensus.cc:3037] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:42.669322 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9dee4b3b14264f3aad0896f629ab5b27 consensus state has no leader
I20260808 13:35:42.673605 8152 raft_consensus.cc:482] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:42.674150 8152 raft_consensus.cc:504] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:42.678442 5771 leader_election.cc:341] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Not found: Tablet not found: d8b07b7d31ef41e484389c0f007598f0
I20260808 13:35:42.680433 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 58b61e17eada43b5a6ae8a664483588d (DEFAULT_TABLE table=default.Table_1955ad58908247a3b3002dc6a8ff2336 [id=85520e0aaa3942268225520f28506cd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.681582 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b61e17eada43b5a6ae8a664483588d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.683058 8152 leader_election.cc:288] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:42.692751 5829 tablet_service.cc:1467] Processing CreateTablet for tablet 26ce3623d547476684782fd395fb8fcf (DEFAULT_TABLE table=default.Table_cfa1c9cda5c140d2920999367a8c9509 [id=7c5d4bd23c38487dad01123b0e852dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.685316 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 58b61e17eada43b5a6ae8a664483588d (DEFAULT_TABLE table=default.Table_1955ad58908247a3b3002dc6a8ff2336 [id=85520e0aaa3942268225520f28506cd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.694504 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ce3623d547476684782fd395fb8fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.699893 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b61e17eada43b5a6ae8a664483588d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.699759 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 58b61e17eada43b5a6ae8a664483588d (DEFAULT_TABLE table=default.Table_1955ad58908247a3b3002dc6a8ff2336 [id=85520e0aaa3942268225520f28506cd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.700768 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b61e17eada43b5a6ae8a664483588d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.699759 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 26ce3623d547476684782fd395fb8fcf (DEFAULT_TABLE table=default.Table_cfa1c9cda5c140d2920999367a8c9509 [id=7c5d4bd23c38487dad01123b0e852dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.703706 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ce3623d547476684782fd395fb8fcf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:42.719178 5283 consensus_peers.cc:487] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:42.720039 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
W20260808 13:35:42.720759 5772 leader_election.cc:341] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: d8b07b7d31ef41e484389c0f007598f0
I20260808 13:35:42.727953 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 26ce3623d547476684782fd395fb8fcf (DEFAULT_TABLE table=default.Table_cfa1c9cda5c140d2920999367a8c9509 [id=7c5d4bd23c38487dad01123b0e852dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.729172 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ce3623d547476684782fd395fb8fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.738462 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 519b2815fd4e4a3691037670c9f61418 (DEFAULT_TABLE table=default.Table_e70822ac71fa4d0e9205367df025ad3a [id=b809e4f7fa074a608bceba5f738df6a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.721133 5772 leader_election.cc:302] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:42.739493 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 519b2815fd4e4a3691037670c9f61418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.743366 8152 raft_consensus.cc:2726] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:42.744457 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 519b2815fd4e4a3691037670c9f61418 (DEFAULT_TABLE table=default.Table_e70822ac71fa4d0e9205367df025ad3a [id=b809e4f7fa074a608bceba5f738df6a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.745596 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 519b2815fd4e4a3691037670c9f61418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.751322 6026 tablet_bootstrap.cc:492] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:42.751870 6026 ts_tablet_manager.cc:1397] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.142s user 0.011s sys 0.000s
I20260808 13:35:42.759860 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 519b2815fd4e4a3691037670c9f61418 (DEFAULT_TABLE table=default.Table_e70822ac71fa4d0e9205367df025ad3a [id=b809e4f7fa074a608bceba5f738df6a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.760903 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 519b2815fd4e4a3691037670c9f61418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.770521 5338 tablet_service.cc:1467] Processing CreateTablet for tablet 2e661b45b9b54a81aaa63b3803636566 (DEFAULT_TABLE table=default.Table_e9d344caa3c2448784d6108b083eca10 [id=6697c24737964dc09cde7d28b2e12447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.771754 5338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e661b45b9b54a81aaa63b3803636566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.774657 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:42.775584 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:42.776309 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:42.777333 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:42.779439 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 2e661b45b9b54a81aaa63b3803636566 (DEFAULT_TABLE table=default.Table_e9d344caa3c2448784d6108b083eca10 [id=6697c24737964dc09cde7d28b2e12447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.780531 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e661b45b9b54a81aaa63b3803636566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.784538 6033 tablet_bootstrap.cc:492] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:42.785050 6033 ts_tablet_manager.cc:1397] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.618s user 0.015s sys 0.012s
I20260808 13:35:42.791273 5337 tablet_service.cc:1467] Processing CreateTablet for tablet 228ad48e495b45a29e96d427b7098497 (DEFAULT_TABLE table=default.Table_a4c8baa577ff42f4b8f9aafb6ec9e6ec [id=d84314027def49d380b8c13a0431f39e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.792418 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228ad48e495b45a29e96d427b7098497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.801085 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 67327badb2524a2c817045a893bcad6e
I20260808 13:35:42.803046 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_7700dbef158a4ddfa8a3122c18e38817"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:42.804546 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7700dbef158a4ddfa8a3122c18e38817 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:42.808753 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 0d8357611ced42db91bd1e416f0e26b3
I20260808 13:35:42.809983 4840 catalog_manager.cc:6565] ReplaceTablet: tablet 0d8357611ced42db91bd1e416f0e26b3 deleted and replaced by tablet 0bf47d3df0ac480bb753d6dc46b4d7db
I20260808 13:35:42.814208 5826 tablet_service.cc:1514] Processing DeleteTablet for tablet 67327badb2524a2c817045a893bcad6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:42 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:42.815204 5838 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d8357611ced42db91bd1e416f0e26b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 0bf47d3df0ac480bb753d6dc46b4d7db) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:42.816963 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d8357611ced42db91bd1e416f0e26b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 0bf47d3df0ac480bb753d6dc46b4d7db) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:42.817391 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 2e661b45b9b54a81aaa63b3803636566 (DEFAULT_TABLE table=default.Table_e9d344caa3c2448784d6108b083eca10 [id=6697c24737964dc09cde7d28b2e12447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.818418 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e661b45b9b54a81aaa63b3803636566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.819391 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d8357611ced42db91bd1e416f0e26b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 0bf47d3df0ac480bb753d6dc46b4d7db) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:42.819437 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 67327badb2524a2c817045a893bcad6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:42 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:42.819517 4840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_41de7b9190a34c0f97e4c0d7fcc14a49" } modify_external_catalogs: true
I20260808 13:35:42.819986 4840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_41de7b9190a34c0f97e4c0d7fcc14a49" } modify_external_catalogs: true
I20260808 13:35:42.828658 6046 tablet_bootstrap.cc:492] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:42.829134 6046 ts_tablet_manager.cc:1397] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.911s user 0.009s sys 0.005s
I20260808 13:35:42.831403 6046 raft_consensus.cc:348] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.832072 6046 raft_consensus.cc:374] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:42.832302 6046 raft_consensus.cc:729] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:42.831774 6033 raft_consensus.cc:348] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.833037 7891 raft_consensus.cc:482] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:42.832965 6046 consensus_queue.cc:260] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.833492 7891 raft_consensus.cc:504] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.836045 7891 leader_election.cc:288] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:42.837188 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52deb2e968c04b99a436e04ae78d9969" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:42.838801 5550 leader_election.cc:341] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:42.839020 7199 ts_tablet_manager.cc:1918] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:42.839501 7199 log.cc:1196] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/11cd8f12cda94c4eb9a848f002a4020b
I20260808 13:35:42.840469 7199 ts_tablet_manager.cc:1939] T 11cd8f12cda94c4eb9a848f002a4020b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:42.841229 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52deb2e968c04b99a436e04ae78d9969" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:42.842885 5551 leader_election.cc:341] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:42.843274 5551 leader_election.cc:302] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:42.846787 7891 raft_consensus.cc:2726] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:42.844040 6046 ts_tablet_manager.cc:1428] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.015s user 0.004s sys 0.004s
I20260808 13:35:42.847427 8141 raft_consensus.cc:504] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.849828 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85b4fd30d6c4903946891a399acf7a0" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:42.850361 5385 raft_consensus.cc:3037] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:42.856865 5385 raft_consensus.cc:2454] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:42.848156 6046 tablet_bootstrap.cc:492] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:42.860605 5551 leader_election.cc:302] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
W20260808 13:35:42.868319 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 0d8357611ced42db91bd1e416f0e26b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d8357611ced42db91bd1e416f0e26b3 already in progress: creating tablet
I20260808 13:35:42.832610 6033 raft_consensus.cc:374] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:42.871073 6033 raft_consensus.cc:729] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:42.875077 6033 consensus_queue.cc:260] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.879470 7891 raft_consensus.cc:2781] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:42.881883 7891 raft_consensus.cc:686] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:42.871942 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 388) took 3740 ms. Trace:
I20260808 13:35:42.882637 7199 rpcz_store.cc:276] 0808 13:35:39.131441 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.131809 (+ 368us) service_pool.cc:227] Handling call
0808 13:35:41.812784 (+2680975us) ts_tablet_manager.cc:1145] Deleting tablet 11cd8f12cda94c4eb9a848f002a4020b
0808 13:35:41.812820 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.228584 (+415764us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:42.781681 (+553097us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:42.871872 (+ 90191us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":24852,"tablet-delete.queue_time_us":2651326}
I20260808 13:35:42.882751 7891 consensus_queue.cc:237] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.883193 4836 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]). Expected version 1 got 0
I20260808 13:35:42.815311 6026 raft_consensus.cc:348] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.887888 6026 raft_consensus.cc:374] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:42.888186 6026 raft_consensus.cc:729] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:42.888922 6026 consensus_queue.cc:260] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:42.890192 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 11cd8f12cda94c4eb9a848f002a4020b (table default.Table_2b5f163ace454f5e87d04751846435e8 [id=d5b2a1e2eebe4f1695a98f926a8cb628]) successfully deleted
I20260808 13:35:42.890240 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 390) took 3730 ms. Trace:
I20260808 13:35:42.893385 8141 leader_election.cc:288] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:42.894531 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85b4fd30d6c4903946891a399acf7a0" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:42.895187 5877 raft_consensus.cc:3037] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:42.902057 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 0d8357611ced42db91bd1e416f0e26b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d8357611ced42db91bd1e416f0e26b3 already in progress: creating tablet
I20260808 13:35:42.905542 6046 tablet_bootstrap.cc:654] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:42.909853 5877 raft_consensus.cc:2454] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:42.914335 8152 consensus_queue.cc:1035] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:42.915700 5772 consensus_peers.cc:487] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 369a942248bd4cec9ee9170fe55601a3. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:42.918246 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 67327badb2524a2c817045a893bcad6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67327badb2524a2c817045a893bcad6e already in progress: creating tablet
I20260808 13:35:42.891408 7199 rpcz_store.cc:276] 0808 13:35:39.160069 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.161539 (+ 1470us) service_pool.cc:227] Handling call
0808 13:35:42.889877 (+3728338us) ts_tablet_manager.cc:1145] Deleting tablet fc9b2e69fe27431dac9eb5ecb9e3e23e
0808 13:35:42.889968 (+ 91us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.890185 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3724884}
I20260808 13:35:42.886148 6033 ts_tablet_manager.cc:1428] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.101s user 0.007s sys 0.002s
I20260808 13:35:42.940569 6033 tablet_bootstrap.cc:492] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:42.941188 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 389) took 3796 ms. Trace:
I20260808 13:35:42.942025 7199 rpcz_store.cc:276] 0808 13:35:39.145055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.155837 (+ 10782us) service_pool.cc:227] Handling call
0808 13:35:42.924811 (+3768974us) ts_tablet_manager.cc:1145] Deleting tablet e00f2e0b98af431d99894e8778257bc4
0808 13:35:42.940889 (+ 16078us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.941132 (+ 243us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3757534}
I20260808 13:35:42.945011 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 391) took 3782 ms. Trace:
I20260808 13:35:42.947021 7199 rpcz_store.cc:276] 0808 13:35:39.163069 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.163380 (+ 311us) service_pool.cc:227] Handling call
0808 13:35:42.944725 (+3781345us) ts_tablet_manager.cc:1145] Deleting tablet 9423b380a62c467895e7fdff14756586
0808 13:35:42.944777 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.944964 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3757485}
I20260808 13:35:42.947391 6026 ts_tablet_manager.cc:1428] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.195s user 0.005s sys 0.001s
I20260808 13:35:42.948298 6026 tablet_bootstrap.cc:492] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:42.950927 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet 67327badb2524a2c817045a893bcad6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:42 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:42.951614 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 392) took 3603 ms. Trace:
I20260808 13:35:42.951891 7199 rpcz_store.cc:276] 0808 13:35:39.347827 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.367001 (+ 19174us) service_pool.cc:227] Handling call
0808 13:35:42.951165 (+3584164us) ts_tablet_manager.cc:1145] Deleting tablet d0767140736242f18ef4689b112deefb
0808 13:35:42.951223 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.951568 (+ 345us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3583375}
I20260808 13:35:42.953051 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 394) took 3425 ms. Trace:
I20260808 13:35:42.953341 7199 rpcz_store.cc:276] 0808 13:35:39.527619 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.551597 (+ 23978us) service_pool.cc:227] Handling call
0808 13:35:42.952661 (+3401064us) ts_tablet_manager.cc:1145] Deleting tablet 6e3ad9ca75b541798bc04d7701128faa
0808 13:35:42.952740 (+ 79us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.953005 (+ 265us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3400291}
I20260808 13:35:42.955813 5325 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:46324 (request call id 41) took 1412 ms. Trace:
I20260808 13:35:42.956156 5325 rpcz_store.cc:276] 0808 13:35:41.543425 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.543578 (+ 153us) service_pool.cc:227] Handling call
0808 13:35:42.955747 (+1412169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1792}
W20260808 13:35:42.960533 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 67327badb2524a2c817045a893bcad6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67327badb2524a2c817045a893bcad6e already in progress: creating tablet
I20260808 13:35:42.965862 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 395) took 2986 ms. Trace:
I20260808 13:35:42.966176 7199 rpcz_store.cc:276] 0808 13:35:39.979586 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.979937 (+ 351us) service_pool.cc:227] Handling call
0808 13:35:42.965410 (+2985473us) ts_tablet_manager.cc:1145] Deleting tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f
0808 13:35:42.965470 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.965794 (+ 324us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2984886}
I20260808 13:35:42.967572 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 397) took 2925 ms. Trace:
I20260808 13:35:42.967861 7199 rpcz_store.cc:276] 0808 13:35:40.042559 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.050724 (+ 8165us) service_pool.cc:227] Handling call
0808 13:35:42.967160 (+2916436us) ts_tablet_manager.cc:1145] Deleting tablet 0b316390787f4170956f1c47a525d430
0808 13:35:42.967215 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.967527 (+ 312us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2915880}
I20260808 13:35:42.968938 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 402) took 2512 ms. Trace:
I20260808 13:35:42.969208 7199 rpcz_store.cc:276] 0808 13:35:40.456204 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.456563 (+ 359us) service_pool.cc:227] Handling call
0808 13:35:42.968630 (+2512067us) ts_tablet_manager.cc:1145] Deleting tablet 264ee45f8dfa4a05b0e5e812ce1b622c
0808 13:35:42.968686 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.968883 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2511468}
W20260808 13:35:42.973374 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet fc9b2e69fe27431dac9eb5ecb9e3e23e because the tablet was not found. No further retry: Not found: Tablet not found: fc9b2e69fe27431dac9eb5ecb9e3e23e
W20260808 13:35:42.974005 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet e00f2e0b98af431d99894e8778257bc4 because the tablet was not found. No further retry: Not found: Tablet not found: e00f2e0b98af431d99894e8778257bc4
W20260808 13:35:42.975381 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 9423b380a62c467895e7fdff14756586 because the tablet was not found. No further retry: Not found: Tablet not found: 9423b380a62c467895e7fdff14756586
W20260808 13:35:42.976006 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet d0767140736242f18ef4689b112deefb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0767140736242f18ef4689b112deefb already in progress: creating tablet
W20260808 13:35:42.976610 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 6e3ad9ca75b541798bc04d7701128faa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e3ad9ca75b541798bc04d7701128faa already in progress: creating tablet
W20260808 13:35:42.977149 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01ca2e4d6a2a4e2bb2dd63af8c125e9f already in progress: creating tablet
W20260808 13:35:42.979122 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 0b316390787f4170956f1c47a525d430 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b316390787f4170956f1c47a525d430 already in progress: creating tablet
W20260808 13:35:42.979910 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 264ee45f8dfa4a05b0e5e812ce1b622c because the tablet was not found. No further retry: Not found: Tablet not found: 264ee45f8dfa4a05b0e5e812ce1b622c
I20260808 13:35:42.980459 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 401) took 2525 ms. Trace:
I20260808 13:35:42.980831 7199 rpcz_store.cc:276] 0808 13:35:40.455494 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.461155 (+ 5661us) service_pool.cc:227] Handling call
0808 13:35:42.970778 (+2509623us) ts_tablet_manager.cc:1145] Deleting tablet 69642fdba1f145438b04eae9f8154101
0808 13:35:42.970841 (+ 63us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:42.971025 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2508965}
I20260808 13:35:42.983810 4840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 8bac8adbffa84d8e8f405305b92c4500
I20260808 13:35:42.989902 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 228ad48e495b45a29e96d427b7098497 (DEFAULT_TABLE table=default.Table_a4c8baa577ff42f4b8f9aafb6ec9e6ec [id=d84314027def49d380b8c13a0431f39e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:42.991014 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228ad48e495b45a29e96d427b7098497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.991541 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bac8adbffa84d8e8f405305b92c4500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:42 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:42.992647 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 228ad48e495b45a29e96d427b7098497 (DEFAULT_TABLE table=default.Table_a4c8baa577ff42f4b8f9aafb6ec9e6ec [id=d84314027def49d380b8c13a0431f39e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:42.993248 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 8bac8adbffa84d8e8f405305b92c4500 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8bac8adbffa84d8e8f405305b92c4500 already in progress: creating tablet
I20260808 13:35:42.993920 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228ad48e495b45a29e96d427b7098497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:42.994184 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bac8adbffa84d8e8f405305b92c4500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:42 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:42.996763 7891 raft_consensus.cc:482] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:42.997234 7891 raft_consensus.cc:504] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:42.983836 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 69642fdba1f145438b04eae9f8154101 because the tablet was not found. No further retry: Not found: Tablet not found: 69642fdba1f145438b04eae9f8154101
I20260808 13:35:43.000398 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bac8adbffa84d8e8f405305b92c4500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:42 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:43.000801 7891 leader_election.cc:288] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.001608 6046 tablet_bootstrap.cc:492] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:43.001585 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff163c36c6d42238cce2396102f9a68" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:43.002868 5550 leader_election.cc:341] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.005041 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff163c36c6d42238cce2396102f9a68" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:43.006280 5551 leader_election.cc:341] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.006806 5551 leader_election.cc:302] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.007936 7891 raft_consensus.cc:2726] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:42.989151 7199 tablet_replica.cc:330] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:43.016038 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 0553d86a78fe4cc2896efe960ba51a27 (DEFAULT_TABLE table=default.Table_d20dd87e2f844e739e8f18ec0be12963 [id=9477c15dbac14aa3a08e0b4a82097291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:43.019479 6046 ts_tablet_manager.cc:1397] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.171s user 0.012s sys 0.000s
W20260808 13:35:43.020679 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 8bac8adbffa84d8e8f405305b92c4500 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8bac8adbffa84d8e8f405305b92c4500 already in progress: creating tablet
I20260808 13:35:43.021528 7199 raft_consensus.cc:2229] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:43.021997 7199 raft_consensus.cc:2258] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:43.023509 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
I20260808 13:35:43.023869 6046 raft_consensus.cc:348] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.024502 6046 raft_consensus.cc:374] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.024744 6046 raft_consensus.cc:729] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.026058 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 0553d86a78fe4cc2896efe960ba51a27 (DEFAULT_TABLE table=default.Table_d20dd87e2f844e739e8f18ec0be12963 [id=9477c15dbac14aa3a08e0b4a82097291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:43.027177 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0553d86a78fe4cc2896efe960ba51a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:43.032272 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 0df909324ec7453f8ecd627490802c35 (DEFAULT_TABLE table=default.Table_db9615359c8549588ff9f1d557189a0d [id=29ca8ec3c8a14e8fb3db662cbc5b6907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:43.033301 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0df909324ec7453f8ecd627490802c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:43.032927 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_4d655ec59e044877882b5cbb4a10a597"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:43.034320 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d655ec59e044877882b5cbb4a10a597 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:43.039927 7199 ts_tablet_manager.cc:1905] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:43.041415 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 0553d86a78fe4cc2896efe960ba51a27 (DEFAULT_TABLE table=default.Table_d20dd87e2f844e739e8f18ec0be12963 [id=9477c15dbac14aa3a08e0b4a82097291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:43.042649 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0553d86a78fe4cc2896efe960ba51a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:43.048722 6046 consensus_queue.cc:260] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.056216 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 0df909324ec7453f8ecd627490802c35 (DEFAULT_TABLE table=default.Table_db9615359c8549588ff9f1d557189a0d [id=29ca8ec3c8a14e8fb3db662cbc5b6907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:43.058863 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 1950961bbaee468c8b16a764cf37f969 (DEFAULT_TABLE table=default.Table_5668bb8655e949f494b5c2e988ae33d5 [id=951c9bdf311a42a0b6fd001ef990e319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:43.067272 6026 tablet_bootstrap.cc:654] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.068396 6046 ts_tablet_manager.cc:1428] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.049s user 0.005s sys 0.001s
I20260808 13:35:43.076052 5829 tablet_service.cc:1467] Processing CreateTablet for tablet 1950961bbaee468c8b16a764cf37f969 (DEFAULT_TABLE table=default.Table_5668bb8655e949f494b5c2e988ae33d5 [id=951c9bdf311a42a0b6fd001ef990e319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:43.077210 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1950961bbaee468c8b16a764cf37f969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:43.083351 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_6c7876f6c0ea460d8fa5cd00a7baee97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:43.087630 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c7876f6c0ea460d8fa5cd00a7baee97 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:43.090348 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 0df909324ec7453f8ecd627490802c35 (DEFAULT_TABLE table=default.Table_db9615359c8549588ff9f1d557189a0d [id=29ca8ec3c8a14e8fb3db662cbc5b6907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:43.091536 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0df909324ec7453f8ecd627490802c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:43.122505 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0553d86a78fe4cc2896efe960ba51a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:43.152581 6033 tablet_bootstrap.cc:654] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.156054 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1950961bbaee468c8b16a764cf37f969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:43.159301 7891 raft_consensus.cc:482] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.159777 7891 raft_consensus.cc:504] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.160485 7199 ts_tablet_manager.cc:1918] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:43.160897 7199 log.cc:1196] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/d8b07b7d31ef41e484389c0f007598f0
I20260808 13:35:43.161576 7199 ts_tablet_manager.cc:1939] T d8b07b7d31ef41e484389c0f007598f0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:43.161861 7891 leader_election.cc:288] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.163681 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:43.165630 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:43.167754 5551 leader_election.cc:341] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:43.169799 5550 leader_election.cc:341] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.170296 6026 tablet_bootstrap.cc:492] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:43.170161 5550 leader_election.cc:302] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.170735 6026 ts_tablet_manager.cc:1397] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.223s user 0.006s sys 0.007s
I20260808 13:35:43.173024 7891 raft_consensus.cc:2726] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:43.173103 6026 raft_consensus.cc:348] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.173801 6026 raft_consensus.cc:374] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.174049 6026 raft_consensus.cc:729] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.174671 6026 consensus_queue.cc:260] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.175809 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0df909324ec7453f8ecd627490802c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:43.176527 6046 tablet_bootstrap.cc:492] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:43.176810 6026 ts_tablet_manager.cc:1428] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260808 13:35:43.177680 6026 tablet_bootstrap.cc:492] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:43.162406 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 405) took 2424 ms. Trace:
I20260808 13:35:43.178897 7199 rpcz_store.cc:276] 0808 13:35:40.738021 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.738160 (+ 139us) service_pool.cc:227] Handling call
0808 13:35:42.984111 (+2245951us) ts_tablet_manager.cc:1145] Deleting tablet d8b07b7d31ef41e484389c0f007598f0
0808 13:35:42.989020 (+ 4909us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:43.138455 (+149435us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:43.160238 (+ 21783us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:43.162349 (+ 2111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2235104}
I20260808 13:35:43.180609 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 404) took 2446 ms. Trace:
I20260808 13:35:43.180922 7199 rpcz_store.cc:276] 0808 13:35:40.734534 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.734777 (+ 243us) service_pool.cc:227] Handling call
0808 13:35:43.180201 (+2445424us) ts_tablet_manager.cc:1145] Deleting tablet 107309c7ada246b6a15a784d7a8c399d
0808 13:35:43.180262 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:43.180553 (+ 291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2429885}
I20260808 13:35:43.182135 7199 tablet_replica.cc:330] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:43.182965 7199 raft_consensus.cc:2229] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:43.183465 7199 raft_consensus.cc:2258] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:43.183463 8141 raft_consensus.cc:482] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.183885 8141 raft_consensus.cc:504] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.185598 7199 ts_tablet_manager.cc:1905] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:43.185771 8141 leader_election.cc:288] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.187592 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594afd4174644c218951b9df82360bfe" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:43.189747 5550 leader_election.cc:341] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.192075 6026 tablet_bootstrap.cc:654] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.196081 6033 tablet_bootstrap.cc:492] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:43.196504 6033 ts_tablet_manager.cc:1397] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.256s user 0.005s sys 0.009s
I20260808 13:35:43.198796 6033 raft_consensus.cc:348] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.199421 6033 raft_consensus.cc:374] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.199672 6033 raft_consensus.cc:729] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.201781 6033 consensus_queue.cc:260] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.203781 6033 ts_tablet_manager.cc:1428] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260808 13:35:43.209111 6026 tablet_bootstrap.cc:492] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:43.209529 6026 ts_tablet_manager.cc:1397] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.000s
I20260808 13:35:43.211922 6026 raft_consensus.cc:348] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.212641 6026 raft_consensus.cc:374] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.212937 6026 raft_consensus.cc:729] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.215029 6033 tablet_bootstrap.cc:492] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:43.215543 6026 consensus_queue.cc:260] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.219257 7199 ts_tablet_manager.cc:1918] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:43.219623 7199 log.cc:1196] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/f0ba43d1479a4f9e9d08bdfa29e7157b
I20260808 13:35:43.220103 6033 tablet_bootstrap.cc:654] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.220345 7199 ts_tablet_manager.cc:1939] T f0ba43d1479a4f9e9d08bdfa29e7157b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:43.221176 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 406) took 2312 ms. Trace:
I20260808 13:35:43.221480 7199 rpcz_store.cc:276] 0808 13:35:40.909286 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.909424 (+ 138us) service_pool.cc:227] Handling call
0808 13:35:43.181981 (+2272557us) ts_tablet_manager.cc:1145] Deleting tablet f0ba43d1479a4f9e9d08bdfa29e7157b
0808 13:35:43.182036 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:43.201426 (+ 19390us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:43.219044 (+ 17618us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:43.221126 (+ 2082us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2272041}
I20260808 13:35:43.223939 6026 ts_tablet_manager.cc:1428] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260808 13:35:43.225888 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 407) took 2087 ms. Trace:
I20260808 13:35:43.226188 7199 rpcz_store.cc:276] 0808 13:35:41.138686 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.140782 (+ 2096us) service_pool.cc:227] Handling call
0808 13:35:43.225453 (+2084671us) ts_tablet_manager.cc:1145] Deleting tablet aa493b8652e149f988d41bee811f6a2f
0808 13:35:43.225508 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:43.225817 (+ 309us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2084025}
I20260808 13:35:43.230015 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 408) took 2063 ms. Trace:
I20260808 13:35:43.230315 7199 rpcz_store.cc:276] 0808 13:35:41.166352 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.170577 (+ 4225us) service_pool.cc:227] Handling call
0808 13:35:43.229618 (+2059041us) ts_tablet_manager.cc:1145] Deleting tablet 2483b3320e2e49db96d39254dd5755aa
0808 13:35:43.229672 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:43.229966 (+ 294us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2058346}
I20260808 13:35:43.231845 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 409) took 1996 ms. Trace:
I20260808 13:35:43.232133 7199 rpcz_store.cc:276] 0808 13:35:41.235968 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.248234 (+ 12266us) service_pool.cc:227] Handling call
0808 13:35:43.231462 (+1983228us) ts_tablet_manager.cc:1145] Deleting tablet 290081bbe4164acd8808041b76441f0d
0808 13:35:43.231509 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:43.231795 (+ 286us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1982516}
I20260808 13:35:43.233546 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 410) took 1961 ms. Trace:
I20260808 13:35:43.233834 7199 rpcz_store.cc:276] 0808 13:35:41.272403 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.280717 (+ 8314us) service_pool.cc:227] Handling call
0808 13:35:43.233196 (+1952479us) ts_tablet_manager.cc:1145] Deleting tablet c8cc2a1b87b14d29838bbbe3de44d2a1
0808 13:35:43.233243 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:43.233497 (+ 254us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1947630}
I20260808 13:35:43.235298 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 411) took 1779 ms. Trace:
I20260808 13:35:43.235555 7199 rpcz_store.cc:276] 0808 13:35:41.456415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.458744 (+ 2329us) service_pool.cc:227] Handling call
0808 13:35:43.234936 (+1776192us) ts_tablet_manager.cc:1145] Deleting tablet de6800eceb96475284fdbffb4d23e5e3
0808 13:35:43.234990 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:43.235252 (+ 262us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1535012}
I20260808 13:35:43.237213 7199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 413) took 1382 ms. Trace:
I20260808 13:35:43.237483 7199 rpcz_store.cc:276] 0808 13:35:41.855309 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.855474 (+ 165us) service_pool.cc:227] Handling call
0808 13:35:43.236885 (+1381411us) ts_tablet_manager.cc:1145] Deleting tablet 809049a771224082a16c270d01ec4bc7
0808 13:35:43.236930 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:43.237173 (+ 243us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1380723}
I20260808 13:35:43.237735 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594afd4174644c218951b9df82360bfe" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:43.239025 6046 tablet_bootstrap.cc:654] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:43.239063 5551 leader_election.cc:341] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.239450 5551 leader_election.cc:302] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.241137 8141 raft_consensus.cc:2726] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:43.243274 6026 tablet_bootstrap.cc:492] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:43.248953 6026 tablet_bootstrap.cc:654] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.260092 6026 tablet_bootstrap.cc:492] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:43.260563 6026 ts_tablet_manager.cc:1397] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.000s
I20260808 13:35:43.263187 6026 raft_consensus.cc:348] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.263859 6026 raft_consensus.cc:374] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.264313 6026 raft_consensus.cc:729] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.264969 6026 consensus_queue.cc:260] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.267621 6026 ts_tablet_manager.cc:1428] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 13:35:43.268566 6026 tablet_bootstrap.cc:492] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:43.269162 7891 raft_consensus.cc:482] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.269562 7891 raft_consensus.cc:504] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.271457 7891 leader_election.cc:288] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.272936 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a73e98a8b20144508a5a8d451f4bb302" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:43.273164 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a73e98a8b20144508a5a8d451f4bb302" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:43.274142 5550 leader_election.cc:341] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:43.274516 5551 leader_election.cc:341] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.274962 5551 leader_election.cc:302] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.275703 7891 raft_consensus.cc:2726] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:43.282436 6026 tablet_bootstrap.cc:654] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.308074 6033 tablet_bootstrap.cc:492] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:43.308655 6033 ts_tablet_manager.cc:1397] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.094s user 0.010s sys 0.004s
I20260808 13:35:43.322822 8048 raft_consensus.cc:482] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.323455 8048 raft_consensus.cc:504] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.326916 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5663fa6a1bb148c3afc9340477b05a4b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:43.327693 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5663fa6a1bb148c3afc9340477b05a4b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:43.327740 5630 raft_consensus.cc:2454] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:43.334077 6033 raft_consensus.cc:348] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.340353 6033 raft_consensus.cc:374] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.340631 6033 raft_consensus.cc:729] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.341260 6033 consensus_queue.cc:260] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.342083 6026 tablet_bootstrap.cc:492] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:43.342545 6026 ts_tablet_manager.cc:1397] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.074s user 0.023s sys 0.038s
I20260808 13:35:43.345351 6026 raft_consensus.cc:348] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.346100 6026 raft_consensus.cc:374] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.346412 6026 raft_consensus.cc:729] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.347285 6026 consensus_queue.cc:260] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.349586 6026 ts_tablet_manager.cc:1428] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 13:35:43.350476 6026 tablet_bootstrap.cc:492] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:43.355967 6026 tablet_bootstrap.cc:654] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.367476 6033 ts_tablet_manager.cc:1428] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.059s user 0.009s sys 0.000s
I20260808 13:35:43.368634 6033 tablet_bootstrap.cc:492] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:43.391841 4821 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4561c9af06e14ed8a2ec4c7285e2ad99 (table default.Table_0a849c19427549aea0eabc1e0324725f [id=f58086e90fc34ca3a5d31417a82dad6c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4561c9af06e14ed8a2ec4c7285e2ad99 consensus state has no leader
I20260808 13:35:43.395354 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:43.395311 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 1950961bbaee468c8b16a764cf37f969 (DEFAULT_TABLE table=default.Table_5668bb8655e949f494b5c2e988ae33d5 [id=951c9bdf311a42a0b6fd001ef990e319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:43.396157 4838 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:43.397722 4838 catalog_manager.cc:5495] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:43.402714 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet d8b07b7d31ef41e484389c0f007598f0 (table default.Table_0bde50fe531b4d5a8d76821fde24d7b1 [id=3bec6c1bb9db43d2a1e00a267f12b8ee]) successfully deleted
W20260808 13:35:43.404503 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 107309c7ada246b6a15a784d7a8c399d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 107309c7ada246b6a15a784d7a8c399d already in progress: creating tablet
I20260808 13:35:43.405834 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet f0ba43d1479a4f9e9d08bdfa29e7157b (table default.Table_8d93597641274e5594c62600af306920 [id=b5733467d1904db5b6df9045484d6a94]) successfully deleted
W20260808 13:35:43.407226 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet aa493b8652e149f988d41bee811f6a2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa493b8652e149f988d41bee811f6a2f already in progress: creating tablet
I20260808 13:35:43.409757 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1950961bbaee468c8b16a764cf37f969. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:43.396237 5283 leader_election.cc:341] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.417689 5283 leader_election.cc:302] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.396574 8048 leader_election.cc:288] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.426192 8146 raft_consensus.cc:2781] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:43.426745 8146 raft_consensus.cc:482] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:43.427090 8146 raft_consensus.cc:3037] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:43.419049 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 90098.2us
I20260808 13:35:43.435503 8146 raft_consensus.cc:504] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.437467 8146 leader_election.cc:288] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.437980 7891 raft_consensus.cc:482] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.438443 7891 raft_consensus.cc:504] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.440353 7891 leader_election.cc:288] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.442904 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5663fa6a1bb148c3afc9340477b05a4b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:43.447233 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249dcaf769934b75945d79d827a96be6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:43.448676 5550 leader_election.cc:341] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.452184 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249dcaf769934b75945d79d827a96be6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:43.452486 5283 leader_election.cc:341] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:43.457420 5551 leader_election.cc:341] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.457785 5551 leader_election.cc:302] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.460211 8141 raft_consensus.cc:2726] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:43.470858 4819 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 63703us
W20260808 13:35:43.471654 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 2483b3320e2e49db96d39254dd5755aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2483b3320e2e49db96d39254dd5755aa already in progress: creating tablet
W20260808 13:35:43.472610 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 290081bbe4164acd8808041b76441f0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 290081bbe4164acd8808041b76441f0d already in progress: creating tablet
W20260808 13:35:43.473644 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet c8cc2a1b87b14d29838bbbe3de44d2a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8cc2a1b87b14d29838bbbe3de44d2a1 already in progress: creating tablet
W20260808 13:35:43.474753 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet de6800eceb96475284fdbffb4d23e5e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de6800eceb96475284fdbffb4d23e5e3 already in progress: creating tablet
I20260808 13:35:43.485492 6046 tablet_bootstrap.cc:492] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:43.486115 6046 ts_tablet_manager.cc:1397] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.310s user 0.019s sys 0.012s
W20260808 13:35:43.491760 8034 lock_manager.cc:586] Waited 3 seconds to obtain row lock on key '<redacted>' (key hash 5193375188705914492) tablet 00000000000000000000000000000000 cur holder 0x7b5800062400
I20260808 13:35:43.491339 6046 raft_consensus.cc:348] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.492254 6046 raft_consensus.cc:374] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.492571 6046 raft_consensus.cc:729] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:35:43.500532 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 809049a771224082a16c270d01ec4bc7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 809049a771224082a16c270d01ec4bc7 already in progress: creating tablet
W20260808 13:35:43.501562 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 67327badb2524a2c817045a893bcad6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67327badb2524a2c817045a893bcad6e already in progress: creating tablet
W20260808 13:35:43.502671 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 0d8357611ced42db91bd1e416f0e26b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d8357611ced42db91bd1e416f0e26b3 already in progress: creating tablet
W20260808 13:35:43.503731 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 8bac8adbffa84d8e8f405305b92c4500 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8bac8adbffa84d8e8f405305b92c4500 already in progress: creating tablet
W20260808 13:35:43.515117 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 79303.5us
I20260808 13:35:43.518478 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5663fa6a1bb148c3afc9340477b05a4b" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:43.519055 5644 raft_consensus.cc:3037] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:43.525957 5644 raft_consensus.cc:2454] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:43.531669 5883 raft_consensus.cc:3037] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:43.536080 6033 tablet_bootstrap.cc:654] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.537065 6026 tablet_bootstrap.cc:492] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:43.537513 6026 ts_tablet_manager.cc:1397] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.187s user 0.044s sys 0.113s
I20260808 13:35:43.540028 6026 raft_consensus.cc:348] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.540683 6026 raft_consensus.cc:374] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.540951 6026 raft_consensus.cc:729] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.541534 6026 consensus_queue.cc:260] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.543035 6046 consensus_queue.cc:260] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.543989 6026 ts_tablet_manager.cc:1428] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 13:35:43.505764 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:43.547113 4841 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:43.547412 5285 leader_election.cc:302] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.548233 8048 raft_consensus.cc:2781] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:43.548669 8048 raft_consensus.cc:686] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:43.549436 8048 consensus_queue.cc:237] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.557394 6026 tablet_bootstrap.cc:492] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:43.577494 6026 tablet_bootstrap.cc:654] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.593349 4840 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:43.594345 4840 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:43.597164 6046 ts_tablet_manager.cc:1428] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.111s user 0.007s sys 0.000s
I20260808 13:35:43.605629 6046 tablet_bootstrap.cc:492] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:43.622915 7891 raft_consensus.cc:482] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.623402 7891 raft_consensus.cc:504] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.625854 6026 tablet_bootstrap.cc:492] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:43.626366 6026 ts_tablet_manager.cc:1397] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.069s user 0.013s sys 0.018s
I20260808 13:35:43.627174 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04d3dd028da4e7f86689e819548d0c5" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:43.628400 5551 leader_election.cc:341] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.630010 7891 leader_election.cc:288] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.595971 4840 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
I20260808 13:35:43.636358 4840 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447]). Expected version 1 got 0
I20260808 13:35:43.638944 6026 raft_consensus.cc:348] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.639658 6026 raft_consensus.cc:374] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.639905 6026 raft_consensus.cc:729] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.640640 6026 consensus_queue.cc:260] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.643488 6026 ts_tablet_manager.cc:1428] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260808 13:35:43.643352 4840 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 52deb2e968c04b99a436e04ae78d9969 (table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445]). Expected version 2 got 0
I20260808 13:35:43.656042 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04d3dd028da4e7f86689e819548d0c5" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:43.656915 5462 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
W20260808 13:35:43.660501 5550 leader_election.cc:341] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.660970 5550 leader_election.cc:302] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.662217 8141 raft_consensus.cc:2726] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:43.665516 6026 tablet_bootstrap.cc:492] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:43.674747 6046 tablet_bootstrap.cc:654] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.675622 6026 tablet_bootstrap.cc:654] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.683104 8141 consensus_queue.cc:1035] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:43.685065 6033 tablet_bootstrap.cc:492] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:43.685469 6033 ts_tablet_manager.cc:1397] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.317s user 0.011s sys 0.006s
I20260808 13:35:43.689976 6033 raft_consensus.cc:348] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.690583 6033 raft_consensus.cc:374] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.690871 6033 raft_consensus.cc:729] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.690862 8141 raft_consensus.cc:482] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.691262 8141 raft_consensus.cc:504] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.693094 8141 leader_election.cc:288] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.693712 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c2b34c95244472beec43643e60ff5f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:43.694808 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c2b34c95244472beec43643e60ff5f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:43.694952 5551 leader_election.cc:341] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:43.696055 5550 leader_election.cc:341] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.696460 5550 leader_election.cc:302] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.696292 6033 consensus_queue.cc:260] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.697921 7891 raft_consensus.cc:2726] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:43.704366 6026 tablet_bootstrap.cc:492] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:43.704519 7957 raft_consensus.cc:482] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.704792 6026 ts_tablet_manager.cc:1397] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.002s
I20260808 13:35:43.704917 7957 raft_consensus.cc:504] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.705569 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:43.706264 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:43.707929 7957 leader_election.cc:288] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:43.709539 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "275b6aeaad044ea1a16ecd82f63dc59f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:43.709936 6026 raft_consensus.cc:348] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.710741 6026 raft_consensus.cc:374] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.710928 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "275b6aeaad044ea1a16ecd82f63dc59f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:43.711053 6026 raft_consensus.cc:729] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.712446 5772 leader_election.cc:302] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:43.714067 8203 raft_consensus.cc:2726] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:43.731810 6033 ts_tablet_manager.cc:1428] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.046s user 0.003s sys 0.003s
I20260808 13:35:43.732822 6033 tablet_bootstrap.cc:492] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:43.751379 6026 consensus_queue.cc:260] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.754068 6026 ts_tablet_manager.cc:1428] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.049s user 0.010s sys 0.001s
I20260808 13:35:43.744628 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:43.755925 6026 tablet_bootstrap.cc:492] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:43.760950 6026 tablet_bootstrap.cc:654] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.769996 6046 tablet_bootstrap.cc:492] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:43.770567 6046 ts_tablet_manager.cc:1397] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.165s user 0.010s sys 0.002s
I20260808 13:35:43.773080 6046 raft_consensus.cc:348] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.773733 6046 raft_consensus.cc:374] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.755815 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:43.799743 6026 tablet_bootstrap.cc:492] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:43.800190 6026 ts_tablet_manager.cc:1397] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.000s
I20260808 13:35:43.802418 6026 raft_consensus.cc:348] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.803094 6026 raft_consensus.cc:374] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.803333 6026 raft_consensus.cc:729] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.803890 6026 consensus_queue.cc:260] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.809437 6033 tablet_bootstrap.cc:654] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.819113 6026 ts_tablet_manager.cc:1428] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260808 13:35:43.823349 8141 consensus_queue.cc:1035] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:43.778549 4839 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f85b4fd30d6c4903946891a399acf7a0 (table default.Table_7da887b39f244cf0b44cbe9fa3b0eae2 [id=ea2ba343c17d403d8cb0bf9d80bee453]). Expected version 1 got 0
I20260808 13:35:43.827112 4842 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_2d75fe6a74ad43789e515c67b5167ab3 [id=29db8794594e4221be8974258b5127e0] alter complete (version 1)
I20260808 13:35:43.829176 4833 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 475) took 1875 ms. Trace:
I20260808 13:35:43.829488 4833 rpcz_store.cc:276] 0808 13:35:41.954103 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.954751 (+ 648us) service_pool.cc:227] Handling call
0808 13:35:41.967121 (+ 12370us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff7d6 00007f227bd45634 00007f227bd455c2 00007f227bd454f1 00007f227bd4546c 00007f227bd44722 00007f2279fde94c 00007f2279fddd86
0808 13:35:41.967845 (+ 724us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:41.968536 (+ 691us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:41.969162 (+ 626us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:41.969382 (+ 220us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:42.121326 (+151944us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:43.828976 (+1707650us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:42.079397 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:42.079766 (+ 369us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:42.079789 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:42.079803 (+ 14us) tablet.cc:660] Decoding operations
0808 13:35:42.080056 (+ 253us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:42.080089 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:42.080109 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:42.080250 (+ 141us) tablet.cc:699] Row locks acquired
0808 13:35:42.080263 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:42.080363 (+ 100us) write_op.cc:270] Start()
0808 13:35:42.080439 (+ 76us) write_op.cc:276] Timestamp: P: 1786196142080328 usec, L: 0
0808 13:35:42.080455 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:42.080928 (+ 473us) log.cc:842] Serialized 405 byte log entry
0808 13:35:42.109972 (+ 29044us) op_driver.cc:444] REPLICATION: finished
0808 13:35:42.114322 (+ 4350us) write_op.cc:301] APPLY: starting
0808 13:35:42.114426 (+ 104us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:42.114597 (+ 171us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:42.119907 (+ 5310us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:42.120230 (+ 323us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:42.120451 (+ 221us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:42.120525 (+ 74us) write_op.cc:312] APPLY: finished
0808 13:35:42.120763 (+ 238us) log.cc:842] Serialized 23 byte log entry
0808 13:35:42.120961 (+ 198us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:42.121051 (+ 90us) write_op.cc:454] Released schema lock
0808 13:35:42.121099 (+ 48us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:42.121324 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:42.121355 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":26478976,"child_traces":[["op",{"apply.queue_time_us":4130,"apply.run_cpu_time_us":2243,"apply.run_wall_time_us":7226,"num_ops":1,"prepare.queue_time_us":108730,"prepare.run_cpu_time_us":1777,"prepare.run_wall_time_us":1776,"raft.queue_time_us":5675,"raft.run_cpu_time_us":9,"raft.run_wall_time_us":9,"replication_time_us":29431}]]}
I20260808 13:35:43.773980 6046 raft_consensus.cc:729] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.839828 6046 consensus_queue.cc:260] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.848320 6046 ts_tablet_manager.cc:1428] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.077s user 0.005s sys 0.001s
I20260808 13:35:43.849192 6046 tablet_bootstrap.cc:492] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:43.857245 6033 tablet_bootstrap.cc:492] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:43.863732 6033 ts_tablet_manager.cc:1397] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.131s user 0.009s sys 0.001s
I20260808 13:35:43.865850 6033 raft_consensus.cc:348] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:43.893540 8207 raft_consensus.cc:482] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.894129 8207 raft_consensus.cc:504] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.895447 8221 tablet.cc:1722] T d9268f5f6dee4112932e106010a3c2be P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:43.897670 8207 leader_election.cc:288] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.911592 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa86f45770d44e7b993b240607ef2fc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:43.913002 5550 leader_election.cc:341] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.866448 6033 raft_consensus.cc:374] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:43.918789 6033 raft_consensus.cc:729] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:43.918918 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa86f45770d44e7b993b240607ef2fc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:43.919574 6033 consensus_queue.cc:260] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:43.855063 4837 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc] alter complete (version 1)
I20260808 13:35:43.931598 6026 tablet_bootstrap.cc:492] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:43.932116 8141 raft_consensus.cc:482] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.932533 8141 raft_consensus.cc:504] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:43.932915 5551 leader_election.cc:341] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.933300 5551 leader_election.cc:302] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.933945 8211 raft_consensus.cc:2726] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:43.942234 4842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 52) took 3340 ms (client timeout 1000 ms). Trace:
I20260808 13:35:43.942459 6026 tablet_bootstrap.cc:654] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:43.942924 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8cf06462484c99a860762a82f7ad64 with delete_type TABLET_DATA_DELETED (replaced by tablet 16736500f94c43d3a3163f61b7606aad) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:43.945320 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:22 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:43.945996 8141 leader_election.cc:288] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:43.942581 4842 rpcz_store.cc:269] 0808 13:35:40.601383 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.601497 (+ 114us) service_pool.cc:227] Handling call
0808 13:35:43.942123 (+3340626us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:41.052761 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:41.053116 (+ 355us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:41.053139 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:41.053152 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:41.053402 (+ 250us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:41.053434 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:41.053455 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:41.053583 (+ 128us) tablet.cc:699] Row locks acquired
0808 13:35:41.053595 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:41.053692 (+ 97us) write_op.cc:270] Start()
0808 13:35:41.053763 (+ 71us) write_op.cc:276] Timestamp: P: 1786196141053607 usec, L: 0
0808 13:35:41.053778 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:41.054263 (+ 485us) log.cc:842] Serialized 527 byte log entry
0808 13:35:41.091243 (+ 36980us) op_driver.cc:444] REPLICATION: finished
0808 13:35:41.091519 (+ 276us) write_op.cc:301] APPLY: starting
0808 13:35:41.091572 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:41.091667 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:41.091755 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:41.091943 (+ 188us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:41.092068 (+ 125us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:41.092114 (+ 46us) write_op.cc:312] APPLY: finished
0808 13:35:41.092285 (+ 171us) log.cc:842] Serialized 23 byte log entry
0808 13:35:41.092434 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:41.092496 (+ 62us) write_op.cc:454] Released schema lock
0808 13:35:41.092525 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:41.092714 (+ 189us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:41.092738 (+ 24us) write_op.cc:454] Released schema lock
Related trace 'op':
0808 13:35:41.186368 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:41.186789 (+ 421us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:41.186816 (+ 27us) write_op.cc:435] Acquired schema lock
0808 13:35:41.186829 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:41.230367 (+ 43538us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:41.230409 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:41.230432 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:41.230572 (+ 140us) tablet.cc:699] Row locks acquired
0808 13:35:41.230587 (+ 15us) write_op.cc:260] PREPARE: finished
0808 13:35:41.230716 (+ 129us) write_op.cc:270] Start()
0808 13:35:41.230795 (+ 79us) write_op.cc:276] Timestamp: P: 1786196141230655 usec, L: 0
0808 13:35:41.230811 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:41.231319 (+ 508us) log.cc:842] Serialized 356 byte log entry
0808 13:35:41.247451 (+ 16132us) op_driver.cc:444] REPLICATION: finished
0808 13:35:41.257387 (+ 9936us) write_op.cc:301] APPLY: starting
0808 13:35:41.257445 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:41.257531 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:41.257636 (+ 105us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:41.257779 (+ 143us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:41.257909 (+ 130us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:41.257951 (+ 42us) write_op.cc:312] APPLY: finished
0808 13:35:41.258117 (+ 166us) log.cc:842] Serialized 23 byte log entry
0808 13:35:41.258256 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:41.258322 (+ 66us) write_op.cc:454] Released schema lock
0808 13:35:41.258351 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:41.258513 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:41.258539 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13178,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":141,"apply.run_cpu_time_us":1364,"apply.run_wall_time_us":1363,"num_ops":1,"prepare.queue_time_us":33518,"prepare.run_cpu_time_us":1687,"prepare.run_wall_time_us":1686,"replication_time_us":37379}],["op",{"apply.queue_time_us":9766,"apply.run_cpu_time_us":1334,"apply.run_wall_time_us":1334,"num_ops":1,"prepare.queue_time_us":68202,"prepare.run_cpu_time_us":2070,"prepare.run_wall_time_us":51658,"raft.queue_time_us":3767,"raft.run_cpu_time_us":559,"raft.run_wall_time_us":549,"replication_time_us":16560}]]}
I20260808 13:35:43.965783 4842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_d39fd423849c45c69b7338e8c6a87c30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:43.967947 4842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d39fd423849c45c69b7338e8c6a87c30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:43.973913 4837 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 58) took 3721 ms (client timeout 1000 ms). Trace:
W20260808 13:35:43.974330 4837 rpcz_store.cc:269] 0808 13:35:40.252584 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.270981 (+ 18397us) service_pool.cc:227] Handling call
0808 13:35:40.406200 (+135219us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:43.973819 (+3567619us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:40.630277 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:40.630657 (+ 380us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:40.630679 (+ 22us) write_op.cc:435] Acquired schema lock
0808 13:35:40.630692 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:40.630967 (+ 275us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:40.631010 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:40.631032 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:40.631181 (+ 149us) tablet.cc:699] Row locks acquired
0808 13:35:40.631212 (+ 31us) write_op.cc:260] PREPARE: finished
0808 13:35:40.631344 (+ 132us) write_op.cc:270] Start()
0808 13:35:40.631426 (+ 82us) write_op.cc:276] Timestamp: P: 1786196140631301 usec, L: 0
0808 13:35:40.631441 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:40.631935 (+ 494us) log.cc:842] Serialized 356 byte log entry
0808 13:35:40.642350 (+ 10415us) op_driver.cc:444] REPLICATION: finished
0808 13:35:40.642759 (+ 409us) write_op.cc:301] APPLY: starting
0808 13:35:40.642827 (+ 68us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:40.642969 (+ 142us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:40.643142 (+ 173us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:40.643339 (+ 197us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:40.643519 (+ 180us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:40.643577 (+ 58us) write_op.cc:312] APPLY: finished
0808 13:35:40.643786 (+ 209us) log.cc:842] Serialized 23 byte log entry
0808 13:35:40.643965 (+ 179us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:40.644045 (+ 80us) write_op.cc:454] Released schema lock
0808 13:35:40.644084 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:40.644344 (+ 260us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:40.644373 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":93813,"reported_tablets":12,"spinlock_wait_cycles":12114688,"child_traces":[["op",{"apply.queue_time_us":245,"apply.run_cpu_time_us":1781,"apply.run_wall_time_us":1781,"num_ops":1,"prepare.queue_time_us":92,"prepare.run_cpu_time_us":1947,"prepare.run_wall_time_us":1947,"raft.queue_time_us":1208,"raft.run_cpu_time_us":565,"raft.run_wall_time_us":565,"replication_time_us":10838}]]}
I20260808 13:35:43.978487 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:43.979000 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0b76c4fdd94a6e9d9bb7709ff1b4be" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:43.979583 8141 raft_consensus.cc:482] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.980044 8141 raft_consensus.cc:504] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:43.981984 8141 leader_election.cc:288] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:43.928826 8203 raft_consensus.cc:482] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:43.947052 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0b76c4fdd94a6e9d9bb7709ff1b4be" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:43.970063 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cd8f12cda94c4eb9a848f002a4020b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:37 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:43.985333 8203 raft_consensus.cc:504] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:43.991372 5550 leader_election.cc:341] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.992727 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e4af746774747a70d425068e8dd63" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:43.993952 8203 leader_election.cc:288] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:43.993988 5550 leader_election.cc:341] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.996500 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e4af746774747a70d425068e8dd63" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:43.996614 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8cf06462484c99a860762a82f7ad64" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:43.997716 5551 leader_election.cc:341] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.998128 5551 leader_election.cc:302] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:43.998811 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8cf06462484c99a860762a82f7ad64" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:43.999171 5551 leader_election.cc:341] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:43.999361 5644 raft_consensus.cc:2454] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:43.999508 5551 leader_election.cc:302] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:44.000509 8187 raft_consensus.cc:482] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.000552 5772 leader_election.cc:302] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:44.001430 8141 raft_consensus.cc:2726] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:44.001113 8187 raft_consensus.cc:504] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.001998 8141 raft_consensus.cc:2726] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:44.002759 8203 raft_consensus.cc:2781] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:44.003240 8203 raft_consensus.cc:482] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:44.003247 8187 leader_election.cc:288] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.003584 8203 raft_consensus.cc:3037] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.004886 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa59a18ae624bc8975bee2c460895a7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:44.005446 5644 raft_consensus.cc:2454] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:44.005488 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa59a18ae624bc8975bee2c460895a7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:44.006624 5285 leader_election.cc:302] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
W20260808 13:35:44.006973 5771 leader_election.cc:341] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:44.008293 5283 leader_election.cc:341] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.010497 8048 raft_consensus.cc:2781] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:44.010670 8223 tablet_replica.cc:330] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:44.010913 8048 raft_consensus.cc:482] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:44.011262 8048 raft_consensus.cc:3037] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.011570 8223 raft_consensus.cc:2229] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:44.012027 8223 raft_consensus.cc:2258] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:44.013840 8223 ts_tablet_manager.cc:1905] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:44.019029 8225 tablet.cc:1722] T d9268f5f6dee4112932e106010a3c2be P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:44.023152 8229 tablet_replica.cc:330] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:44.027937 8229 raft_consensus.cc:2229] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:44.028489 8229 raft_consensus.cc:2258] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:44.030807 8229 ts_tablet_manager.cc:1905] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:44.037082 8203 raft_consensus.cc:504] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.038054 8048 raft_consensus.cc:504] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.042219 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet f0ba43d1479a4f9e9d08bdfa29e7157b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:44.042435 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:44.042237 5829 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d8d5a3ca724141b58e4e2cea23be68 with delete_type TABLET_DATA_DELETED (replaced by tablet 14a9373000e24e75858e6d7457587264) from {username='slave'} at 127.0.0.1:54830
W20260808 13:35:44.051524 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 11cd8f12cda94c4eb9a848f002a4020b because the tablet was not found. No further retry: Not found: Tablet not found: 11cd8f12cda94c4eb9a848f002a4020b
I20260808 13:35:44.052562 8226 tablet_replica.cc:330] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:44.052450 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8cf06462484c99a860762a82f7ad64" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:44.053508 8226 raft_consensus.cc:2229] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
W20260808 13:35:44.053825 5771 leader_election.cc:341] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.054591 8226 raft_consensus.cc:2258] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:44.057017 8226 ts_tablet_manager.cc:1905] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:44.061471 8222 tablet.cc:1722] T d9268f5f6dee4112932e106010a3c2be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260808 13:35:44.063331 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8cf06462484c99a860762a82f7ad64" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:44.070232 8229 ts_tablet_manager.cc:1918] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:44.080420 8229 log.cc:1196] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/8b57966a9b7b4673ab9f98720cfd456d
I20260808 13:35:44.081460 8229 ts_tablet_manager.cc:1939] T 8b57966a9b7b4673ab9f98720cfd456d P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:44.085281 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 8b57966a9b7b4673ab9f98720cfd456d (table default.Table_d70de767ab2548b4a9e45418eefc9a2b [id=10bb97e006904dc9b34ebedad3ec13de]) successfully deleted
I20260808 13:35:44.041842 6046 tablet_bootstrap.cc:654] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.048113 4837 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_7da887b39f244cf0b44cbe9fa3b0eae2" } new_table_name: "default.Table_f042821be4a8421ea3f64f33a071b22d" modify_external_catalogs: true
W20260808 13:35:44.098222 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 43) took 3167 ms (client timeout 1000 ms). Trace:
W20260808 13:35:44.098658 4832 rpcz_store.cc:269] 0808 13:35:40.930571 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:40.930859 (+ 288us) service_pool.cc:227] Handling call
0808 13:35:40.940714 (+ 9855us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:44.098002 (+3157288us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:41.864065 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:41.864483 (+ 418us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:41.864505 (+ 22us) write_op.cc:435] Acquired schema lock
0808 13:35:41.864527 (+ 22us) tablet.cc:660] Decoding operations
0808 13:35:41.864810 (+ 283us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:41.864867 (+ 57us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:41.864895 (+ 28us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:41.865051 (+ 156us) tablet.cc:699] Row locks acquired
0808 13:35:41.865063 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:41.865169 (+ 106us) write_op.cc:270] Start()
0808 13:35:41.865245 (+ 76us) write_op.cc:276] Timestamp: P: 1786196141865119 usec, L: 0
0808 13:35:41.865261 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:41.943284 (+ 78023us) log.cc:842] Serialized 527 byte log entry
0808 13:35:41.943955 (+ 671us) spinlock_profiling.cc:243] Waited 77.4 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:42.003224 (+ 59269us) op_driver.cc:444] REPLICATION: finished
0808 13:35:42.005133 (+ 1909us) write_op.cc:301] APPLY: starting
0808 13:35:42.005230 (+ 97us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:42.005430 (+ 200us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:42.005502 (+ 72us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:42.006023 (+ 521us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:42.006315 (+ 292us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:42.006407 (+ 92us) write_op.cc:312] APPLY: finished
0808 13:35:42.006811 (+ 404us) log.cc:842] Serialized 23 byte log entry
0808 13:35:42.007060 (+ 249us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:42.007147 (+ 87us) write_op.cc:454] Released schema lock
0808 13:35:42.007205 (+ 58us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:42.007500 (+ 295us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:42.007541 (+ 41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9002,"reported_tablets":27,"spinlock_wait_cycles":12674944,"child_traces":[["op",{"apply.queue_time_us":119,"apply.run_cpu_time_us":2539,"apply.run_wall_time_us":2737,"num_ops":1,"prepare.queue_time_us":142881,"prepare.run_cpu_time_us":3517,"prepare.run_wall_time_us":80093,"raft.queue_time_us":11325,"raft.run_cpu_time_us":1697,"raft.run_wall_time_us":1698,"replication_time_us":137890,"spinlock_wait_cycles":177906432}]]}
I20260808 13:35:44.104885 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa59a18ae624bc8975bee2c460895a7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:44.105453 5644 raft_consensus.cc:3037] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.105400 8048 leader_election.cc:288] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.114254 5644 raft_consensus.cc:2454] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:44.120491 5285 leader_election.cc:302] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:44.122515 8187 raft_consensus.cc:2781] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:44.123080 8187 raft_consensus.cc:686] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:44.124091 8187 consensus_queue.cc:237] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.134596 4842 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 486) took 1415 ms. Trace:
I20260808 13:35:44.134971 4842 rpcz_store.cc:276] 0808 13:35:42.719116 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:43.965265 (+1246149us) service_pool.cc:227] Handling call
0808 13:35:43.968366 (+ 3101us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:44.080838 (+112472us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:44.133698 (+ 52860us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:44.133725 (+ 27us) write_op.cc:454] Released schema lock
0808 13:35:44.133937 (+ 212us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:44.134268 (+ 331us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:44.134525 (+ 257us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:44.115757 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:44.116113 (+ 356us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:44.116136 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:44.116148 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:44.116479 (+ 331us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:44.116518 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:44.116537 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:44.116719 (+ 182us) tablet.cc:699] Row locks acquired
0808 13:35:44.116732 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:44.116860 (+ 128us) write_op.cc:270] Start()
0808 13:35:44.116939 (+ 79us) write_op.cc:276] Timestamp: P: 1786196144116795 usec, L: 0
0808 13:35:44.116955 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:44.117418 (+ 463us) log.cc:842] Serialized 433 byte log entry
0808 13:35:44.126373 (+ 8955us) op_driver.cc:444] REPLICATION: finished
0808 13:35:44.126709 (+ 336us) write_op.cc:301] APPLY: starting
0808 13:35:44.126767 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:44.126869 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:44.126985 (+ 116us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:44.127404 (+ 419us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:44.127532 (+ 128us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:44.127571 (+ 39us) write_op.cc:312] APPLY: finished
0808 13:35:44.127739 (+ 168us) log.cc:842] Serialized 29 byte log entry
0808 13:35:44.127871 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:44.127935 (+ 64us) write_op.cc:454] Released schema lock
0808 13:35:44.127964 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":143,"apply.run_cpu_time_us":1422,"apply.run_wall_time_us":1422,"num_ops":2,"prepare.queue_time_us":87,"prepare.run_cpu_time_us":1933,"prepare.run_wall_time_us":5709,"raft.queue_time_us":252,"raft.run_cpu_time_us":567,"raft.run_wall_time_us":566,"replication_time_us":9346}]]}
W20260808 13:35:44.067350 5772 leader_election.cc:341] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: Tablet not RUNNING: STOPPED
I20260808 13:35:44.166102 5772 leader_election.cc:302] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:44.166756 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 99436.4us
I20260808 13:35:44.168262 7957 raft_consensus.cc:2726] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 13:35:44.169302 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6db6d1f1b6454336958e53566b8d48e0" } modify_external_catalogs: true
I20260808 13:35:44.169723 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa59a18ae624bc8975bee2c460895a7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:44.169857 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_6db6d1f1b6454336958e53566b8d48e0" } modify_external_catalogs: true
W20260808 13:35:44.173025 5283 leader_election.cc:341] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.174906 8223 ts_tablet_manager.cc:1918] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:44.175354 8223 log.cc:1196] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/7e8cf06462484c99a860762a82f7ad64
I20260808 13:35:44.082545 8203 leader_election.cc:288] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:44.176110 8223 ts_tablet_manager.cc:1939] T 7e8cf06462484c99a860762a82f7ad64 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:44.083950 6033 ts_tablet_manager.cc:1428] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.118s user 0.004s sys 0.000s
I20260808 13:35:44.096119 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 328d361c336848689c6906ca6fe2ccc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:35 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:44.134176 5878 raft_consensus.cc:3037] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.197165 6026 tablet_bootstrap.cc:492] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:44.198272 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 7e8cf06462484c99a860762a82f7ad64 (table default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]) successfully deleted
I20260808 13:35:44.197635 6026 ts_tablet_manager.cc:1397] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.266s user 0.055s sys 0.058s
I20260808 13:35:44.200917 6026 raft_consensus.cc:348] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.201541 6026 raft_consensus.cc:374] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.201803 6026 raft_consensus.cc:729] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.216527 6046 tablet_bootstrap.cc:492] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:44.216975 6046 ts_tablet_manager.cc:1397] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.368s user 0.004s sys 0.004s
I20260808 13:35:44.219923 6046 raft_consensus.cc:348] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.220566 6046 raft_consensus.cc:374] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.220798 6046 raft_consensus.cc:729] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.221354 6046 consensus_queue.cc:260] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.226975 6026 consensus_queue.cc:260] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.235396 6026 ts_tablet_manager.cc:1428] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.036s user 0.003s sys 0.002s
I20260808 13:35:44.235672 8187 raft_consensus.cc:482] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.236347 6026 tablet_bootstrap.cc:492] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:44.236112 8187 raft_consensus.cc:504] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.238751 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "851a0b374a854efd87dd66e39b9df2d5" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:44.239012 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "851a0b374a854efd87dd66e39b9df2d5" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:44.239305 5644 raft_consensus.cc:2454] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:44.239576 5883 raft_consensus.cc:2454] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:44.240597 6033 tablet_bootstrap.cc:492] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:44.240439 5285 leader_election.cc:302] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:44.241194 8048 raft_consensus.cc:2781] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:44.241513 8048 raft_consensus.cc:482] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:44.241822 8048 raft_consensus.cc:3037] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.208225 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet d439b27c3ad34619972c71d652f1ff10
I20260808 13:35:44.244359 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet d439b27c3ad34619972c71d652f1ff10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:44 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:44.244809 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet d439b27c3ad34619972c71d652f1ff10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:44 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:44.247555 6033 tablet_bootstrap.cc:654] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.248728 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet d439b27c3ad34619972c71d652f1ff10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:44 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:44.249396 4842 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 488) took 1240 ms. Trace:
W20260808 13:35:44.250420 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet d439b27c3ad34619972c71d652f1ff10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d439b27c3ad34619972c71d652f1ff10 already in progress: creating tablet
I20260808 13:35:44.240674 8187 leader_election.cc:288] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.251173 8223 tablet_replica.cc:330] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:44.252127 8223 raft_consensus.cc:2229] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:44.252720 8223 raft_consensus.cc:2258] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:44.254720 8223 ts_tablet_manager.cc:1905] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:44.255504 8048 raft_consensus.cc:504] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.249706 4842 rpcz_store.cc:276] 0808 13:35:43.009438 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:44.168926 (+1159488us) service_pool.cc:227] Handling call
0808 13:35:44.170234 (+ 1308us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:44.170976 (+ 742us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:44.171121 (+ 145us) catalog_manager.cc:2761] Locking tablets
0808 13:35:44.171677 (+ 556us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:44.196507 (+ 24830us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:44.196535 (+ 28us) write_op.cc:454] Released schema lock
0808 13:35:44.196755 (+ 220us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:44.208008 (+ 11253us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:44.208141 (+ 133us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:44.249310 (+ 41169us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:44.177952 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:44.178317 (+ 365us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:44.178340 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:44.178351 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:44.178700 (+ 349us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:44.178739 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:44.178761 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:44.179064 (+ 303us) tablet.cc:699] Row locks acquired
0808 13:35:44.179077 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:44.179186 (+ 109us) write_op.cc:270] Start()
0808 13:35:44.179285 (+ 99us) write_op.cc:276] Timestamp: P: 1786196144179115 usec, L: 0
0808 13:35:44.179303 (+ 18us) op_driver.cc:331] REPLICATION: starting
0808 13:35:44.179800 (+ 497us) log.cc:842] Serialized 725 byte log entry
0808 13:35:44.189319 (+ 9519us) op_driver.cc:444] REPLICATION: finished
0808 13:35:44.189613 (+ 294us) write_op.cc:301] APPLY: starting
0808 13:35:44.189670 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:44.189782 (+ 112us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:44.189880 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:44.190154 (+ 274us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:44.190327 (+ 173us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:44.190370 (+ 43us) write_op.cc:312] APPLY: finished
0808 13:35:44.190549 (+ 179us) log.cc:842] Serialized 29 byte log entry
0808 13:35:44.190720 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:44.190786 (+ 66us) write_op.cc:454] Released schema lock
0808 13:35:44.190815 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":164,"apply.run_cpu_time_us":1376,"apply.run_wall_time_us":1376,"num_ops":2,"prepare.queue_time_us":4370,"prepare.run_cpu_time_us":2110,"prepare.run_wall_time_us":2110,"raft.queue_time_us":1378,"raft.run_cpu_time_us":643,"raft.run_wall_time_us":643,"replication_time_us":9928}]]}
I20260808 13:35:44.269999 4842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_f80c686d970c4c32966a483d401a1e52" } modify_external_catalogs: true
I20260808 13:35:44.270866 8203 raft_consensus.cc:482] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.267426 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "851a0b374a854efd87dd66e39b9df2d5" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:44.271505 5644 raft_consensus.cc:3037] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.271345 8203 raft_consensus.cc:504] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.273221 8203 leader_election.cc:288] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:44.274829 6046 ts_tablet_manager.cc:1428] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.058s user 0.004s sys 0.003s
I20260808 13:35:44.276036 6046 tablet_bootstrap.cc:492] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:44.276990 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362dbd6990e740b39690537af98aa2f8" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:44.278206 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362dbd6990e740b39690537af98aa2f8" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:44.278419 5771 leader_election.cc:341] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.277477 4842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_f80c686d970c4c32966a483d401a1e52" } modify_external_catalogs: true
I20260808 13:35:44.278867 5630 raft_consensus.cc:2454] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:44.279922 6033 tablet_bootstrap.cc:492] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:44.279865 5772 leader_election.cc:302] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:44.280411 6033 ts_tablet_manager.cc:1397] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.040s user 0.010s sys 0.000s
I20260808 13:35:44.281088 8203 raft_consensus.cc:2781] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:44.281435 8203 raft_consensus.cc:482] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:44.281741 8203 raft_consensus.cc:3037] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.282196 8226 ts_tablet_manager.cc:1918] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:44.286661 8187 raft_consensus.cc:482] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.287168 5644 raft_consensus.cc:2454] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:44.287096 8187 raft_consensus.cc:504] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.288151 5285 leader_election.cc:302] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:44.289163 8230 raft_consensus.cc:2781] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:44.289639 8187 leader_election.cc:288] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.290711 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:44.291522 5630 raft_consensus.cc:2454] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:44.292655 6026 tablet_bootstrap.cc:654] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.293161 5285 leader_election.cc:302] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:44.294696 8048 leader_election.cc:288] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.296108 8048 raft_consensus.cc:2781] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:44.296476 8048 raft_consensus.cc:482] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:44.296765 8048 raft_consensus.cc:3037] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.305631 8230 raft_consensus.cc:686] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:44.306558 8230 consensus_queue.cc:237] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.284780 6033 raft_consensus.cc:348] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.315109 6033 raft_consensus.cc:374] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.315050 8226 log.cc:1196] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/8b57966a9b7b4673ab9f98720cfd456d
I20260808 13:35:44.315424 6033 raft_consensus.cc:729] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.316296 8226 ts_tablet_manager.cc:1939] T 8b57966a9b7b4673ab9f98720cfd456d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:44.306736 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "851a0b374a854efd87dd66e39b9df2d5" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:44.319288 5883 raft_consensus.cc:3037] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.320446 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 8b57966a9b7b4673ab9f98720cfd456d (table default.Table_d70de767ab2548b4a9e45418eefc9a2b [id=10bb97e006904dc9b34ebedad3ec13de]) successfully deleted
W20260808 13:35:44.322352 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 56d8d5a3ca724141b58e4e2cea23be68 because the tablet was not found. No further retry: Not found: Tablet not found: 56d8d5a3ca724141b58e4e2cea23be68
W20260808 13:35:44.332700 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet f0ba43d1479a4f9e9d08bdfa29e7157b because the tablet was not found. No further retry: Not found: Tablet not found: f0ba43d1479a4f9e9d08bdfa29e7157b
I20260808 13:35:44.332998 8141 raft_consensus.cc:482] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.333564 5883 raft_consensus.cc:2454] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:44.333632 8141 raft_consensus.cc:504] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.308449 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:44.316195 6033 consensus_queue.cc:260] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.336730 6033 ts_tablet_manager.cc:1428] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.056s user 0.003s sys 0.002s
W20260808 13:35:44.337220 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 328d361c336848689c6906ca6fe2ccc3 because the tablet was not found. No further retry: Not found: Tablet not found: 328d361c336848689c6906ca6fe2ccc3
I20260808 13:35:44.337596 6033 tablet_bootstrap.cc:492] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:44.339231 8106 raft_consensus.cc:482] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.339745 8106 raft_consensus.cc:504] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:44.342257 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet d439b27c3ad34619972c71d652f1ff10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d439b27c3ad34619972c71d652f1ff10 already in progress: creating tablet
I20260808 13:35:44.342728 8141 leader_election.cc:288] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.292956 8203 raft_consensus.cc:504] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.342298 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64888f0d27c94ee7b25c0dda57e766f6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:44.344790 5550 leader_election.cc:341] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.345337 8203 leader_election.cc:288] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:44.346550 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64888f0d27c94ee7b25c0dda57e766f6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:44.348011 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362dbd6990e740b39690537af98aa2f8" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:44.348943 8048 raft_consensus.cc:504] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:44.353428 5771 leader_election.cc:341] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:44.354161 5551 leader_election.cc:341] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.354550 5551 leader_election.cc:302] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:44.355309 8141 raft_consensus.cc:2726] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:44.357208 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:44.359186 5283 leader_election.cc:341] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.359385 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362dbd6990e740b39690537af98aa2f8" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:44.359921 5644 raft_consensus.cc:3037] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.359573 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:44.360612 5630 raft_consensus.cc:2454] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:44.361809 6046 tablet_bootstrap.cc:654] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.357853 8106 leader_election.cc:288] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:44.358367 5771 leader_election.cc:341] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.371002 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:44.371755 5630 raft_consensus.cc:3037] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.372844 5772 leader_election.cc:302] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:44.373855 8106 raft_consensus.cc:2781] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:44.374293 8106 raft_consensus.cc:482] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:44.374835 8106 raft_consensus.cc:3037] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.358052 6033 tablet_bootstrap.cc:654] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.385897 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:44.390909 5283 leader_election.cc:341] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.392063 6046 tablet_bootstrap.cc:492] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:44.398137 5630 raft_consensus.cc:2454] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:44.395303 8048 leader_election.cc:288] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.403816 5285 leader_election.cc:302] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:44.408540 6046 ts_tablet_manager.cc:1397] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.133s user 0.010s sys 0.003s
I20260808 13:35:44.409611 8048 raft_consensus.cc:2781] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:44.410346 8141 raft_consensus.cc:482] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.411347 5644 raft_consensus.cc:2454] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:44.411008 8141 raft_consensus.cc:504] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.412385 5772 leader_election.cc:302] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:44.413220 8203 raft_consensus.cc:2781] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:44.413806 8203 raft_consensus.cc:686] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:44.414831 8203 consensus_queue.cc:237] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.415484 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b2cce690cd462aa58ceee2bd28c638" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:44.417022 5551 leader_election.cc:341] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.417834 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b2cce690cd462aa58ceee2bd28c638" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:44.419111 5550 leader_election.cc:341] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.419450 5550 leader_election.cc:302] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:44.420239 8211 raft_consensus.cc:2726] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:44.420934 8141 leader_election.cc:288] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.421789 8048 raft_consensus.cc:686] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:44.423493 8048 consensus_queue.cc:237] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.427770 6026 tablet_bootstrap.cc:492] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:44.428205 6026 ts_tablet_manager.cc:1397] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.192s user 0.015s sys 0.000s
I20260808 13:35:44.430478 6026 raft_consensus.cc:348] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.431121 6026 raft_consensus.cc:374] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.431375 6026 raft_consensus.cc:729] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.432235 6026 consensus_queue.cc:260] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.427546 6046 raft_consensus.cc:348] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.437862 6046 raft_consensus.cc:374] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.438171 6046 raft_consensus.cc:729] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.438848 6046 consensus_queue.cc:260] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.439823 6026 ts_tablet_manager.cc:1428] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260808 13:35:44.440691 6026 tablet_bootstrap.cc:492] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:44.441118 6046 ts_tablet_manager.cc:1428] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.032s user 0.003s sys 0.002s
I20260808 13:35:44.441938 6046 tablet_bootstrap.cc:492] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:44.448987 8223 ts_tablet_manager.cc:1918] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:44.449365 8223 log.cc:1196] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/196d80d806b0432cacf2ca6aa0da8c6b
W20260808 13:35:44.452418 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2d8e4af746774747a70d425068e8dd63 (table default.Table_938ca078ec5e4411b4be33056272983c [id=95f59bd3447c44c291270c448f53d78b], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2d8e4af746774747a70d425068e8dd63 consensus state has no leader
I20260808 13:35:44.458464 8106 raft_consensus.cc:504] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.461354 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:44.462757 5771 leader_election.cc:341] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.466837 6046 tablet_bootstrap.cc:654] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.462757 6033 tablet_bootstrap.cc:492] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:44.471535 6033 ts_tablet_manager.cc:1397] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.134s user 0.010s sys 0.004s
I20260808 13:35:44.473873 6033 raft_consensus.cc:348] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.476302 6033 raft_consensus.cc:374] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.478796 5948 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:35:44.480515 6033 raft_consensus.cc:729] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.481635 6033 consensus_queue.cc:260] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.485311 6033 ts_tablet_manager.cc:1428] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.013s user 0.004s sys 0.001s
I20260808 13:35:44.500685 8223 ts_tablet_manager.cc:1939] T 196d80d806b0432cacf2ca6aa0da8c6b P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:44.519011 6033 tablet_bootstrap.cc:492] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:44.524201 6033 tablet_bootstrap.cc:654] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.528457 6026 tablet_bootstrap.cc:654] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.535509 8106 leader_election.cc:288] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:44.529410 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 196d80d806b0432cacf2ca6aa0da8c6b (table default.Table_f212f0d1d7294b4f80606ba37c7f19f3 [id=8039834d1c7f4cb6894534c8284b27c1]) successfully deleted
I20260808 13:35:44.540449 6046 tablet_bootstrap.cc:492] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:44.540968 6046 ts_tablet_manager.cc:1397] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.099s user 0.006s sys 0.006s
I20260808 13:35:44.543359 6046 raft_consensus.cc:348] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.544005 6046 raft_consensus.cc:374] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.544243 6046 raft_consensus.cc:729] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.551995 6033 tablet_bootstrap.cc:492] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:44.552390 6033 ts_tablet_manager.cc:1397] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.034s user 0.007s sys 0.001s
I20260808 13:35:44.554522 6033 raft_consensus.cc:348] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.555151 6033 raft_consensus.cc:374] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.555364 6033 raft_consensus.cc:729] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.555985 6033 consensus_queue.cc:260] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:44.556602 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet d439b27c3ad34619972c71d652f1ff10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d439b27c3ad34619972c71d652f1ff10 already in progress: creating tablet
I20260808 13:35:44.558759 6033 ts_tablet_manager.cc:1428] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260808 13:35:44.559607 6033 tablet_bootstrap.cc:492] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:44.562883 6046 consensus_queue.cc:260] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.567350 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:44.567956 5630 raft_consensus.cc:3037] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.574498 5630 raft_consensus.cc:2454] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:44.576296 6046 ts_tablet_manager.cc:1428] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.035s user 0.003s sys 0.002s
I20260808 13:35:44.576730 5772 leader_election.cc:302] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:44.577818 8106 raft_consensus.cc:2781] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:44.578591 8106 raft_consensus.cc:686] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:44.579461 8106 consensus_queue.cc:237] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.584236 6033 tablet_bootstrap.cc:654] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.591486 6026 tablet_bootstrap.cc:492] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:44.591914 6026 ts_tablet_manager.cc:1397] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.151s user 0.009s sys 0.000s
I20260808 13:35:44.595649 6046 tablet_bootstrap.cc:492] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:44.600586 8106 raft_consensus.cc:482] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.601153 8106 raft_consensus.cc:504] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.604040 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:44.607016 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:44.607772 5630 raft_consensus.cc:2379] T 3b1a27f687b54bc592ef4efb72b7247f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:44.608937 5772 leader_election.cc:302] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:44.609628 8203 raft_consensus.cc:3037] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.609728 8106 leader_election.cc:288] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:44.616644 8203 raft_consensus.cc:2726] T 3b1a27f687b54bc592ef4efb72b7247f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:44.617722 6026 raft_consensus.cc:348] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.618438 6026 raft_consensus.cc:374] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.618712 6026 raft_consensus.cc:729] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.619319 6026 consensus_queue.cc:260] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.621413 6026 ts_tablet_manager.cc:1428] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260808 13:35:44.622300 6026 tablet_bootstrap.cc:492] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:44.623204 6046 tablet_bootstrap.cc:654] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:44.632707 5285 consensus_peers.cc:487] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 196d80d806b0432cacf2ca6aa0da8c6b. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:44.653312 8237 consensus_queue.cc:1035] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:44.658730 5720 heartbeater.cc:347] Connected to a master server at 127.4.170.60:37135
I20260808 13:35:44.664386 5883 raft_consensus.cc:3037] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.675100 8211 raft_consensus.cc:482] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.675539 8211 raft_consensus.cc:504] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.677734 8211 leader_election.cc:288] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.678895 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756b5f24ed94a9fbe2a43efc7b8d40d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:44.680907 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756b5f24ed94a9fbe2a43efc7b8d40d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:44.681428 8237 consensus_queue.cc:1035] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:44.682298 5551 leader_election.cc:341] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:44.683382 5550 leader_election.cc:341] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.683718 5550 leader_election.cc:302] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:44.684369 8141 raft_consensus.cc:2726] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:44.686460 6026 tablet_bootstrap.cc:654] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.650167 6033 tablet_bootstrap.cc:492] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:44.701939 6033 ts_tablet_manager.cc:1397] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.142s user 0.011s sys 0.000s
I20260808 13:35:44.707971 6046 tablet_bootstrap.cc:492] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:44.707743 6033 raft_consensus.cc:348] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.708420 6046 ts_tablet_manager.cc:1397] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.113s user 0.007s sys 0.002s
I20260808 13:35:44.708428 6033 raft_consensus.cc:374] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.708683 6033 raft_consensus.cc:729] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.709319 6033 consensus_queue.cc:260] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.709888 8141 raft_consensus.cc:482] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.710374 8141 raft_consensus.cc:504] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.712519 8141 leader_election.cc:288] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.714186 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:44.716327 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:44.717724 5551 leader_election.cc:341] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:44.718828 5550 leader_election.cc:341] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.719158 5550 leader_election.cc:302] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:44.726263 8211 raft_consensus.cc:2726] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:44.737004 6026 tablet_bootstrap.cc:492] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:44.737432 6026 ts_tablet_manager.cc:1397] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.115s user 0.007s sys 0.001s
I20260808 13:35:44.744519 8211 raft_consensus.cc:482] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.744977 8211 raft_consensus.cc:504] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.710520 6046 raft_consensus.cc:348] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.746400 6046 raft_consensus.cc:374] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.746731 6046 raft_consensus.cc:729] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.747732 8211 leader_election.cc:288] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.747674 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bed5e6d1aa49a9b14ba86403c985ce" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:44.752022 6033 ts_tablet_manager.cc:1428] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.050s user 0.006s sys 0.000s
I20260808 13:35:44.739979 6026 raft_consensus.cc:348] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.752931 6026 raft_consensus.cc:374] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.753265 6026 raft_consensus.cc:729] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
W20260808 13:35:44.760455 5551 leader_election.cc:341] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.754417 6026 consensus_queue.cc:260] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.767321 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bed5e6d1aa49a9b14ba86403c985ce" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:44.772305 5550 leader_election.cc:341] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:44.772703 5550 leader_election.cc:302] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:44.773743 8211 raft_consensus.cc:2726] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:44.785707 6046 consensus_queue.cc:260] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.752929 6033 tablet_bootstrap.cc:492] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:44.798856 6026 ts_tablet_manager.cc:1428] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.061s user 0.008s sys 0.000s
I20260808 13:35:44.808645 6046 ts_tablet_manager.cc:1428] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.100s user 0.008s sys 0.000s
I20260808 13:35:44.809700 6046 tablet_bootstrap.cc:492] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:44.820194 6033 tablet_bootstrap.cc:654] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:44.823938 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.495s user 0.044s sys 0.004s
I20260808 13:35:44.829039 6026 tablet_bootstrap.cc:492] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:44.834158 6026 tablet_bootstrap.cc:654] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.843082 6033 tablet_bootstrap.cc:492] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:44.845541 6033 ts_tablet_manager.cc:1397] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.093s user 0.010s sys 0.000s
I20260808 13:35:44.853659 6033 raft_consensus.cc:348] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.854588 6033 raft_consensus.cc:374] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.854867 6033 raft_consensus.cc:729] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.855509 6033 consensus_queue.cc:260] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.864192 6046 tablet_bootstrap.cc:654] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.891798 8141 raft_consensus.cc:482] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:44.891384 7961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e672d7386144b299688abea9c64854c. Latest consensus state: current_term: 3 leader_uuid: "3e672d7386144b299688abea9c64854c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:35:44.892583 7961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:35:44.892230 8141 raft_consensus.cc:504] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.893333 8062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:44.894223 8141 leader_election.cc:288] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.893949 8062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:44.897107 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a27517560d64a3b8da2cf55980e18a8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:44.897706 5883 raft_consensus.cc:2454] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:44.897444 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a27517560d64a3b8da2cf55980e18a8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:44.898300 5381 raft_consensus.cc:2454] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
I20260808 13:35:44.898902 5550 leader_election.cc:302] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:44.899566 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 489 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:44.900466 8141 raft_consensus.cc:2781] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:44.900777 8141 raft_consensus.cc:482] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:44.901080 8141 raft_consensus.cc:3037] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.901937 8062 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:44.903215 6033 ts_tablet_manager.cc:1428] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.057s user 0.006s sys 0.000s
I20260808 13:35:44.903337 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 489 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:44.904475 6033 tablet_bootstrap.cc:492] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:44.905354 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:44.906579 8062 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:44.910812 6026 tablet_bootstrap.cc:492] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:44.911257 6026 ts_tablet_manager.cc:1397] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.082s user 0.011s sys 0.000s
I20260808 13:35:44.913604 6026 raft_consensus.cc:348] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.913712 8141 raft_consensus.cc:504] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.914259 6026 raft_consensus.cc:374] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.919632 8280 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 13:35:44.934885 6033 tablet_bootstrap.cc:654] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:44.938843 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a27517560d64a3b8da2cf55980e18a8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:44.939445 5381 raft_consensus.cc:3037] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.944636 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a27517560d64a3b8da2cf55980e18a8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:44.945220 5883 raft_consensus.cc:3037] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:44.947620 5381 raft_consensus.cc:2454] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:44.948876 5551 leader_election.cc:302] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:44.934592 7662 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" instance_seqno: 1786196103015531) as {username='slave'} at 127.4.170.3:55463; Asking this server to re-register.
I20260808 13:35:44.962772 8141 leader_election.cc:288] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:44.965262 8288 raft_consensus.cc:2781] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:44.965737 8288 raft_consensus.cc:686] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:44.914561 6026 raft_consensus.cc:729] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.966766 8288 consensus_queue.cc:237] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.966981 6026 consensus_queue.cc:260] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:44.969857 5883 raft_consensus.cc:2454] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
I20260808 13:35:44.975785 6033 tablet_bootstrap.cc:492] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:44.976186 6026 ts_tablet_manager.cc:1428] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.065s user 0.006s sys 0.000s
I20260808 13:35:44.976275 6033 ts_tablet_manager.cc:1397] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.072s user 0.007s sys 0.002s
I20260808 13:35:44.978781 5948 heartbeater.cc:464] Registering TS with master...
I20260808 13:35:44.985550 6046 tablet_bootstrap.cc:492] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:44.986021 6046 ts_tablet_manager.cc:1397] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.177s user 0.012s sys 0.000s
I20260808 13:35:44.988758 6046 raft_consensus.cc:348] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.989445 6046 raft_consensus.cc:374] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.989702 6046 raft_consensus.cc:729] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:44.995376 6026 tablet_bootstrap.cc:492] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:44.978519 6033 raft_consensus.cc:348] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:44.998006 6033 raft_consensus.cc:374] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:44.998247 6033 raft_consensus.cc:729] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:45.004294 6033 consensus_queue.cc:260] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.016877 6033 ts_tablet_manager.cc:1428] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.040s user 0.005s sys 0.002s
I20260808 13:35:45.019274 7662 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" instance_seqno: 1786196100161043) as {username='slave'} at 127.4.170.2:44803; Asking this server to re-register.
I20260808 13:35:45.019801 7658 ts_manager.cc:194] Registered new tserver with Master: 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:45.021133 6033 tablet_bootstrap.cc:492] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:45.026921 7659 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "946c9d8462644deba5b45fa724f1923c" instance_seqno: 1786196094059806) as {username='slave'} at 127.4.170.1:49557; Asking this server to re-register.
I20260808 13:35:45.029825 5720 heartbeater.cc:464] Registering TS with master...
I20260808 13:35:45.045845 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:45.048493 5842 tablet_service.cc:1467] Processing CreateTablet for tablet a728cc43df834ae19c27d0c1d9b3a52c (DEFAULT_TABLE table=default.Table_4d655ec59e044877882b5cbb4a10a597 [id=716d7d399d82408fb07943a0e0715f88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.050730 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a728cc43df834ae19c27d0c1d9b3a52c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.056646 5883 raft_consensus.cc:3037] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:45.056613 5595 tablet_service.cc:1467] Processing CreateTablet for tablet a728cc43df834ae19c27d0c1d9b3a52c (DEFAULT_TABLE table=default.Table_4d655ec59e044877882b5cbb4a10a597 [id=716d7d399d82408fb07943a0e0715f88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.057912 5829 tablet_service.cc:1467] Processing CreateTablet for tablet 1f35509a5e584dd6a640eefa9020d8fd (DEFAULT_TABLE table=default.Table_f998dcf284c249d7a6db0e4b803a157a [id=dd464b6ca5eb49499b0921fa2b99cb29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.059245 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a728cc43df834ae19c27d0c1d9b3a52c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.062114 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f35509a5e584dd6a640eefa9020d8fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.053572 5340 tablet_service.cc:1467] Processing CreateTablet for tablet a728cc43df834ae19c27d0c1d9b3a52c (DEFAULT_TABLE table=default.Table_4d655ec59e044877882b5cbb4a10a597 [id=716d7d399d82408fb07943a0e0715f88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.071825 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a728cc43df834ae19c27d0c1d9b3a52c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.060082 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 1f35509a5e584dd6a640eefa9020d8fd (DEFAULT_TABLE table=default.Table_f998dcf284c249d7a6db0e4b803a157a [id=dd464b6ca5eb49499b0921fa2b99cb29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.076213 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f35509a5e584dd6a640eefa9020d8fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.066049 8237 consensus_queue.cc:1035] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260808 13:35:45.086714 6033 tablet_bootstrap.cc:654] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:45.040939 5462 heartbeater.cc:464] Registering TS with master...
I20260808 13:35:45.092368 4842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 521ffd66d22f48188635b85c6e72f650
I20260808 13:35:45.108840 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 521ffd66d22f48188635b85c6e72f650 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:44 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:45.115711 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:45.116990 7659 ts_manager.cc:194] Registered new tserver with Master: 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:45.117511 7658 ts_manager.cc:194] Registered new tserver with Master: 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:45.119035 4842 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 489) took 1721 ms. Trace:
I20260808 13:35:45.121779 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 3d07c40bd01845ef871a9e87ef29e9b7 (DEFAULT_TABLE table=default.Table_e4f01562a52f4079b8b916b057cf8014 [id=76320b3dd2994b4587dc8a6b3a71825d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.122908 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d07c40bd01845ef871a9e87ef29e9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.123952 8230 consensus_queue.cc:1035] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:45.121491 4842 rpcz_store.cc:276] 0808 13:35:43.398076 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:44.269650 (+871574us) service_pool.cc:227] Handling call
0808 13:35:44.278988 (+ 9338us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:44.292132 (+ 13144us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:44.292322 (+ 190us) catalog_manager.cc:2761] Locking tablets
0808 13:35:45.032900 (+740578us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:45.091942 (+ 59042us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:45.092158 (+ 216us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:45.092274 (+ 116us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:45.118776 (+ 26502us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b4c00003800. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f2279f77210 00007f2279f87fb2 00007f2279f88512 00007f2279f4d79b 00007f2279f7a73c 00007f2279f7abfc 00007f227b90bd5b 00007f227e14c79b 00007f227e0f5330 00007f227e0f0da0 00007f227e0e74d0
0808 13:35:45.118952 (+ 176us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:45.038810 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:45.039239 (+ 429us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:45.039265 (+ 26us) write_op.cc:435] Acquired schema lock
0808 13:35:45.039277 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:45.039626 (+ 349us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:45.039671 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:45.039701 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:45.040001 (+ 300us) tablet.cc:699] Row locks acquired
0808 13:35:45.040014 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:45.040144 (+ 130us) write_op.cc:270] Start()
0808 13:35:45.040235 (+ 91us) write_op.cc:276] Timestamp: P: 1786196145040095 usec, L: 0
0808 13:35:45.040250 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:45.040795 (+ 545us) log.cc:842] Serialized 725 byte log entry
0808 13:35:45.047611 (+ 6816us) op_driver.cc:444] REPLICATION: finished
0808 13:35:45.090001 (+ 42390us) write_op.cc:301] APPLY: starting
0808 13:35:45.090059 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:45.090158 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:45.090256 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:45.090577 (+ 321us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:45.090753 (+ 176us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:45.090794 (+ 41us) write_op.cc:312] APPLY: finished
0808 13:35:45.090995 (+ 201us) log.cc:842] Serialized 29 byte log entry
0808 13:35:45.091634 (+ 639us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:45.091706 (+ 72us) write_op.cc:454] Released schema lock
0808 13:35:45.091736 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:45.092756 (+ 1020us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:45.092784 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":20080896,"child_traces":[["op",{"apply.queue_time_us":42210,"apply.run_cpu_time_us":1705,"apply.run_wall_time_us":2975,"num_ops":2,"prepare.queue_time_us":4069,"prepare.run_cpu_time_us":2291,"prepare.run_wall_time_us":2292,"raft.queue_time_us":100,"raft.run_cpu_time_us":694,"raft.run_wall_time_us":694,"replication_time_us":7283,"thread_start_us":45350,"threads_started":1}]]}
I20260808 13:35:45.127187 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_a450e64d7a494bbcb8f35780fda0ccc8 [id=c4338f1c19ec4210a803a1e2aaaa65d4]). Expected version 3 got 2
I20260808 13:35:45.127414 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 1f35509a5e584dd6a640eefa9020d8fd (DEFAULT_TABLE table=default.Table_f998dcf284c249d7a6db0e4b803a157a [id=dd464b6ca5eb49499b0921fa2b99cb29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.127791 4833 catalog_manager.cc:5495] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:45.128525 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f35509a5e584dd6a640eefa9020d8fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.128938 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
I20260808 13:35:45.129927 4833 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet f85b4fd30d6c4903946891a399acf7a0 (table default.Table_7da887b39f244cf0b44cbe9fa3b0eae2 [id=ea2ba343c17d403d8cb0bf9d80bee453]). Expected version 1 got 0
I20260808 13:35:45.130859 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet 521ffd66d22f48188635b85c6e72f650 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:44 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:44.990319 6046 consensus_queue.cc:260] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.139305 6046 ts_tablet_manager.cc:1428] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.153s user 0.006s sys 0.001s
I20260808 13:35:45.145139 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet 521ffd66d22f48188635b85c6e72f650 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:44 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:45.146310 6033 tablet_bootstrap.cc:492] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:45.146224 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 3d07c40bd01845ef871a9e87ef29e9b7 (DEFAULT_TABLE table=default.Table_e4f01562a52f4079b8b916b057cf8014 [id=76320b3dd2994b4587dc8a6b3a71825d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.151688 6033 ts_tablet_manager.cc:1397] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.134s user 0.008s sys 0.001s
I20260808 13:35:45.154666 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 3d07c40bd01845ef871a9e87ef29e9b7 (DEFAULT_TABLE table=default.Table_e4f01562a52f4079b8b916b057cf8014 [id=76320b3dd2994b4587dc8a6b3a71825d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.155727 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d07c40bd01845ef871a9e87ef29e9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.167367 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d07c40bd01845ef871a9e87ef29e9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.172948 6026 tablet_bootstrap.cc:654] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:45.175083 6033 raft_consensus.cc:348] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.175812 6033 raft_consensus.cc:374] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:45.176090 6033 raft_consensus.cc:729] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:45.176817 6033 consensus_queue.cc:260] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:45.178967 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 521ffd66d22f48188635b85c6e72f650 because the tablet was not found. No further retry: Not found: Tablet not found: 521ffd66d22f48188635b85c6e72f650
I20260808 13:35:45.194211 8230 raft_consensus.cc:482] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:45.195108 8230 raft_consensus.cc:504] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:45.199802 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 521ffd66d22f48188635b85c6e72f650 because the tablet was not found. No further retry: Not found: Tablet not found: 521ffd66d22f48188635b85c6e72f650
I20260808 13:35:45.201707 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52deb2e968c04b99a436e04ae78d9969" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:45.202278 5644 raft_consensus.cc:2454] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:45.203788 5838 tablet_service.cc:1467] Processing CreateTablet for tablet deccbee005534189a274ec40bc91048e (DEFAULT_TABLE table=default.Table_bf5259e68f5c4869bac79b5b0a6a0cd0 [id=acbe4578625746948e76ae16ce884569]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.204604 5285 leader_election.cc:302] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:45.205078 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deccbee005534189a274ec40bc91048e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.205484 8048 raft_consensus.cc:2781] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:45.205590 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52deb2e968c04b99a436e04ae78d9969" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:45.205833 8048 raft_consensus.cc:482] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:45.206203 5883 raft_consensus.cc:2454] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:45.206254 8048 raft_consensus.cc:3037] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:45.213032 8230 leader_election.cc:288] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:45.221123 6033 ts_tablet_manager.cc:1428] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.069s user 0.004s sys 0.002s
I20260808 13:35:45.222117 6033 tablet_bootstrap.cc:492] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:45.232725 8106 raft_consensus.cc:482] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:45.233245 8106 raft_consensus.cc:504] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.235704 8288 raft_consensus.cc:482] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:45.236177 8288 raft_consensus.cc:504] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:45.236281 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c2b34c95244472beec43643e60ff5f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:45.236860 8106 leader_election.cc:288] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:45.237731 5771 leader_election.cc:341] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:45.238057 8288 leader_election.cc:288] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:45.240710 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3e9c6770fd402290b45eed85de6f6f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:45.242375 5550 leader_election.cc:341] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:45.243485 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c2b34c95244472beec43643e60ff5f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:45.244045 5630 raft_consensus.cc:2454] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:45.245048 5772 leader_election.cc:302] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:45.245949 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 521ffd66d22f48188635b85c6e72f650 because the tablet was not found. No further retry: Not found: Tablet not found: 521ffd66d22f48188635b85c6e72f650
I20260808 13:35:45.246025 8106 raft_consensus.cc:2781] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:45.246342 8106 raft_consensus.cc:482] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:45.246680 8106 raft_consensus.cc:3037] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:45.247012 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3e9c6770fd402290b45eed85de6f6f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:45.248329 5551 leader_election.cc:341] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:45.248749 5551 leader_election.cc:302] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:45.249786 8141 raft_consensus.cc:2726] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:45.258561 5613 tablet_service.cc:1467] Processing CreateTablet for tablet deccbee005534189a274ec40bc91048e (DEFAULT_TABLE table=default.Table_bf5259e68f5c4869bac79b5b0a6a0cd0 [id=acbe4578625746948e76ae16ce884569]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.257222 5340 tablet_service.cc:1467] Processing CreateTablet for tablet deccbee005534189a274ec40bc91048e (DEFAULT_TABLE table=default.Table_bf5259e68f5c4869bac79b5b0a6a0cd0 [id=acbe4578625746948e76ae16ce884569]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.260243 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deccbee005534189a274ec40bc91048e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.262274 8237 consensus_queue.cc:1035] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:45.140260 6046 tablet_bootstrap.cc:492] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:45.294957 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deccbee005534189a274ec40bc91048e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.293781 8048 raft_consensus.cc:504] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.311376 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52deb2e968c04b99a436e04ae78d9969" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:45.311926 5644 raft_consensus.cc:3037] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:45.315891 8106 raft_consensus.cc:504] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.335518 8237 consensus_queue.cc:1035] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:45.336292 5644 raft_consensus.cc:2454] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:45.337302 5285 leader_election.cc:302] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:45.337999 8237 raft_consensus.cc:2781] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:45.338387 8237 raft_consensus.cc:686] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:45.339545 8237 consensus_queue.cc:237] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.353739 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c2b34c95244472beec43643e60ff5f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:45.353542 5381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c2b34c95244472beec43643e60ff5f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:45.354290 5630 raft_consensus.cc:3037] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:45.355404 5771 leader_election.cc:341] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:45.360805 8048 leader_election.cc:288] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:45.361574 5630 raft_consensus.cc:2454] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:45.362833 5772 leader_election.cc:302] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:45.331183 6033 tablet_bootstrap.cc:700] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent opening tablet: real 0.105s user 0.004s sys 0.000s
I20260808 13:35:45.365808 6033 tablet_bootstrap.cc:654] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:45.372035 5354 tablet_service.cc:1467] Processing CreateTablet for tablet cc4e6d765eb04ca581db353f93d4f5b5 (DEFAULT_TABLE table=default.Table_533328cbcc13486fb2b3c66451f55ac3 [id=e90cdc01582749b092c452f915c314eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.373713 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc4e6d765eb04ca581db353f93d4f5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.373809 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
I20260808 13:35:45.403847 5595 tablet_service.cc:1467] Processing CreateTablet for tablet cc4e6d765eb04ca581db353f93d4f5b5 (DEFAULT_TABLE table=default.Table_533328cbcc13486fb2b3c66451f55ac3 [id=e90cdc01582749b092c452f915c314eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.380875 8312 raft_consensus.cc:2781] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:45.410738 5332 tablet_service.cc:1467] Processing CreateTablet for tablet d392ae0edc0e4976bed7f3c82145f662 (DEFAULT_TABLE table=default.Table_3e5cfb8683d745f08a7738ed4d2cd13a [id=31e622ce08da4164936a264736f75833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.411837 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d392ae0edc0e4976bed7f3c82145f662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.423367 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
W20260808 13:35:45.427139 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 64428.2us
I20260808 13:35:45.428248 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc4e6d765eb04ca581db353f93d4f5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.446075 6046 tablet_bootstrap.cc:654] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:45.447237 5614 tablet_service.cc:1467] Processing CreateTablet for tablet d3c297fe49c74bceb2cfe2fbece2d0ba (DEFAULT_TABLE table=default.Table_6932f06e7e1e4ec79c76ad6efea73902 [id=665fac81069c47c2b2b787ed500d3a14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.447641 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:45.448191 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447]). Expected version 1 got 0
I20260808 13:35:45.448397 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c297fe49c74bceb2cfe2fbece2d0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.449034 4832 catalog_manager.cc:5495] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2). New cstate: current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:45.450186 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f85b4fd30d6c4903946891a399acf7a0 (table default.Table_7da887b39f244cf0b44cbe9fa3b0eae2 [id=ea2ba343c17d403d8cb0bf9d80bee453]). Expected version 1 got 0
I20260808 13:35:45.451683 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 52deb2e968c04b99a436e04ae78d9969 (table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445]). Expected version 2 got 0
I20260808 13:35:45.452492 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 2d8e4af746774747a70d425068e8dd63 (table default.Table_938ca078ec5e4411b4be33056272983c [id=95f59bd3447c44c291270c448f53d78b]). Expected version 2 got 0
I20260808 13:35:45.463452 5613 tablet_service.cc:1467] Processing CreateTablet for tablet d392ae0edc0e4976bed7f3c82145f662 (DEFAULT_TABLE table=default.Table_3e5cfb8683d745f08a7738ed4d2cd13a [id=31e622ce08da4164936a264736f75833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.464866 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d392ae0edc0e4976bed7f3c82145f662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.468657 8048 consensus_queue.cc:1035] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:35:45.480558 4821 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0d0018c5333b4fde8ad4ce874660eb97 (table default.Table_db29cf6251b947269c7a48cfc05ea2e5 [id=86e7a76979764ac09c964217ebe7fae4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0d0018c5333b4fde8ad4ce874660eb97 consensus state has no leader
I20260808 13:35:45.430814 8106 leader_election.cc:288] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:45.493631 8048 raft_consensus.cc:482] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:45.494434 5838 tablet_service.cc:1467] Processing CreateTablet for tablet cc4e6d765eb04ca581db353f93d4f5b5 (DEFAULT_TABLE table=default.Table_533328cbcc13486fb2b3c66451f55ac3 [id=e90cdc01582749b092c452f915c314eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.494316 8048 raft_consensus.cc:504] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.495841 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52deb2e968c04b99a436e04ae78d9969" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:45.496425 5883 raft_consensus.cc:3037] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:45.498653 5842 tablet_service.cc:1467] Processing CreateTablet for tablet d392ae0edc0e4976bed7f3c82145f662 (DEFAULT_TABLE table=default.Table_3e5cfb8683d745f08a7738ed4d2cd13a [id=31e622ce08da4164936a264736f75833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.499123 5829 tablet_service.cc:1467] Processing CreateTablet for tablet 8675dbcac3dc4392be833df89eb3b42c (DEFAULT_TABLE table=default.Table_834da6ca85d64417964c7976b57c1329 [id=ad4e3ed6750e4b7b8262ec862dd1c699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.499658 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d392ae0edc0e4976bed7f3c82145f662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.500222 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8675dbcac3dc4392be833df89eb3b42c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.500414 5843 tablet_service.cc:1467] Processing CreateTablet for tablet d3c297fe49c74bceb2cfe2fbece2d0ba (DEFAULT_TABLE table=default.Table_6932f06e7e1e4ec79c76ad6efea73902 [id=665fac81069c47c2b2b787ed500d3a14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.501313 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c297fe49c74bceb2cfe2fbece2d0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.507449 8048 leader_election.cc:288] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:45.512833 5771 consensus_peers.cc:487] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:45.516958 5285 leader_election.cc:341] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 7e8cf06462484c99a860762a82f7ad64
I20260808 13:35:45.524382 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8cf06462484c99a860762a82f7ad64" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:45.525599 5874 raft_consensus.cc:2379] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:45.546260 5354 tablet_service.cc:1467] Processing CreateTablet for tablet d3c297fe49c74bceb2cfe2fbece2d0ba (DEFAULT_TABLE table=default.Table_6932f06e7e1e4ec79c76ad6efea73902 [id=665fac81069c47c2b2b787ed500d3a14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.523262 8312 raft_consensus.cc:686] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:45.547549 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c297fe49c74bceb2cfe2fbece2d0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.548235 8312 consensus_queue.cc:237] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.569187 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc4e6d765eb04ca581db353f93d4f5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.571252 8323 consensus_queue.cc:1035] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:35:45.572547 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 50531.3us
I20260808 13:35:45.579051 5283 leader_election.cc:302] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:45.581403 8237 raft_consensus.cc:3037] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:45.588058 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 8675dbcac3dc4392be833df89eb3b42c (DEFAULT_TABLE table=default.Table_834da6ca85d64417964c7976b57c1329 [id=ad4e3ed6750e4b7b8262ec862dd1c699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.589521 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8675dbcac3dc4392be833df89eb3b42c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.597301 5381 raft_consensus.cc:3037] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:45.598160 8237 raft_consensus.cc:2726] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:45.602355 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 8675dbcac3dc4392be833df89eb3b42c (DEFAULT_TABLE table=default.Table_834da6ca85d64417964c7976b57c1329 [id=ad4e3ed6750e4b7b8262ec862dd1c699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.603127 6033 tablet_bootstrap.cc:492] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:45.603590 6033 ts_tablet_manager.cc:1397] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.382s user 0.006s sys 0.011s
I20260808 13:35:45.607251 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8675dbcac3dc4392be833df89eb3b42c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.621768 6026 tablet_bootstrap.cc:492] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:45.622335 6026 ts_tablet_manager.cc:1397] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.627s user 0.010s sys 0.015s
I20260808 13:35:45.625999 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:45.626767 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:45.634310 6046 tablet_bootstrap.cc:492] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:45.634815 6046 ts_tablet_manager.cc:1397] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.495s user 0.011s sys 0.010s
I20260808 13:35:45.634377 6026 raft_consensus.cc:348] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:45.636116 6026 raft_consensus.cc:374] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:45.636386 6026 raft_consensus.cc:729] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:45.636960 6026 consensus_queue.cc:260] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:45.637229 6046 raft_consensus.cc:348] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:45.637936 6046 raft_consensus.cc:374] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:45.638208 6046 raft_consensus.cc:729] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:45.638496 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:45.640965 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f85b4fd30d6c4903946891a399acf7a0 (table default.Table_7da887b39f244cf0b44cbe9fa3b0eae2 [id=ea2ba343c17d403d8cb0bf9d80bee453]). Expected version 1 got 0
I20260808 13:35:45.643493 4842 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:45.642580 6026 ts_tablet_manager.cc:1428] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260808 13:35:45.647174 4835 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_a14f7675417240ca88f88da86055bceb [id=16cef8437603476b97a8562edba58a41] alter complete (version 4)
I20260808 13:35:45.649197 6026 tablet_bootstrap.cc:492] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:45.657097 6046 consensus_queue.cc:260] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:45.661834 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a27517560d64a3b8da2cf55980e18a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:45.662896 5598 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8cf06462484c99a860762a82f7ad64 with delete_type TABLET_DATA_DELETED (replaced by tablet 16736500f94c43d3a3163f61b7606aad) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:45.664561 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 362dbd6990e740b39690537af98aa2f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:45.664477 5592 tablet_service.cc:1514] Processing DeleteTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:22 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:45.665364 5616 tablet_service.cc:1514] Processing DeleteTablet for tablet daa59a18ae624bc8975bee2c460895a7 with delete_type TABLET_DATA_DELETED (replaced by tablet cb7f6487e28f4aaa94d448be71f8a3c2) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:45.666018 5610 tablet_service.cc:1514] Processing DeleteTablet for tablet 851a0b374a854efd87dd66e39b9df2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:45.666940 8306 tablet_replica.cc:330] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:45.668548 8306 raft_consensus.cc:2229] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:45.669526 8306 pending_rounds.cc:62] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Trying to abort 1 pending ops.
I20260808 13:35:45.669772 8306 pending_rounds.cc:68] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259409915727872 op_type: NO_OP noop_request { }
W20260808 13:35:45.672359 4838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 53) took 4034 ms (client timeout 1000 ms). Trace:
I20260808 13:35:45.670466 8306 raft_consensus.cc:2866] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:35:45.675431 8306 raft_consensus.cc:2258] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:45.681752 5883 raft_consensus.cc:2454] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:45.686404 8306 ts_tablet_manager.cc:1905] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:45.689189 4839 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a] alter complete (version 1)
I20260808 13:35:45.693614 6033 raft_consensus.cc:348] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.694480 6033 raft_consensus.cc:374] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:45.705950 6046 ts_tablet_manager.cc:1428] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.071s user 0.004s sys 0.003s
I20260808 13:35:45.706987 6046 tablet_bootstrap.cc:492] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:45.694767 6033 raft_consensus.cc:729] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:45.712212 6033 consensus_queue.cc:260] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.714126 6033 ts_tablet_manager.cc:1428] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.110s user 0.005s sys 0.000s
I20260808 13:35:45.715183 6033 tablet_bootstrap.cc:492] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:45.719290 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet f0ba43d1479a4f9e9d08bdfa29e7157b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:45.719281 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 369a942248bd4cec9ee9170fe55601a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:45.720992 8303 tablet_replica.cc:330] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:45.723132 5835 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:45.724304 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 6278446e587b42168d388acdf7ec284e (DEFAULT_TABLE table=default.Table_d99de9b6e1e64b48873233ac5849ea09 [id=88fdd9e2283a4018ad2e1d02dc9cd4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.725284 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6278446e587b42168d388acdf7ec284e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:45.726982 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet f0ba43d1479a4f9e9d08bdfa29e7157b because the tablet was not found. No further retry: Not found: Tablet not found: f0ba43d1479a4f9e9d08bdfa29e7157b
I20260808 13:35:45.738941 4832 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: default.Table_a450e64d7a494bbcb8f35780fda0ccc8 [id=c4338f1c19ec4210a803a1e2aaaa65d4] alter complete (version 3)
I20260808 13:35:45.747363 6046 tablet_bootstrap.cc:654] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:45.752586 4836 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 45) took 3717 ms (client timeout 1000 ms). Trace:
I20260808 13:35:45.753439 8141 raft_consensus.cc:482] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:45.754004 8141 raft_consensus.cc:504] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.756644 8141 leader_election.cc:288] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:45.758935 4840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 54) took 2785 ms (client timeout 1000 ms). Trace:
W20260808 13:35:45.759377 4840 rpcz_store.cc:269] 0808 13:35:42.972737 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:42.985966 (+ 13229us) service_pool.cc:227] Handling call
0808 13:35:45.757543 (+2771577us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":781,"reported_tablets":23}
I20260808 13:35:45.762399 8303 raft_consensus.cc:2229] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:45.763741 8303 raft_consensus.cc:2258] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:45.766189 6033 tablet_bootstrap.cc:654] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:45.766149 8303 ts_tablet_manager.cc:1905] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:45.774930 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 521ffd66d22f48188635b85c6e72f650 (DEFAULT_TABLE table=default.Table_f80c686d970c4c32966a483d401a1e52 [id=0d317758cc2d44d689558f770dae2680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.777184 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 6278446e587b42168d388acdf7ec284e (DEFAULT_TABLE table=default.Table_d99de9b6e1e64b48873233ac5849ea09 [id=88fdd9e2283a4018ad2e1d02dc9cd4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.778424 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6278446e587b42168d388acdf7ec284e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:45.778749 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 122369us
W20260808 13:35:45.771907 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 59) took 4506 ms (client timeout 1000 ms). Trace:
I20260808 13:35:45.781502 6046 tablet_bootstrap.cc:492] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
W20260808 13:35:45.753014 4836 rpcz_store.cc:269] 0808 13:35:42.034624 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:42.034743 (+ 119us) service_pool.cc:227] Handling call
0808 13:35:42.076441 (+ 41698us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:45.714820 (+3638379us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b4400090790. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f2279f5b301 00007f2279f7ab70 00007f227b90bd5b 00007f227e14c79b 00007f227e0f5330 00007f227e0fb083 00007f227e1fe4fe 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec
0808 13:35:45.752381 (+ 37561us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1951,"reported_tablets":27,"spinlock_wait_cycles":154958336}
W20260808 13:35:45.789788 4840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 62) took 1132 ms (client timeout 1000 ms). Trace:
I20260808 13:35:45.781944 6046 ts_tablet_manager.cc:1397] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.075s user 0.004s sys 0.005s
I20260808 13:35:45.795007 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba39706cff65404485959865fff1772a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:45.797591 5550 leader_election.cc:341] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:45.799985 5826 tablet_service.cc:1514] Processing DeleteTablet for tablet f0ba43d1479a4f9e9d08bdfa29e7157b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:45.810380 5826 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b07b7d31ef41e484389c0f007598f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:45.813102 5826 tablet_service.cc:1514] Processing DeleteTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:22 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:45.818871 8312 consensus_queue.cc:1035] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:45.821411 6026 tablet_bootstrap.cc:654] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:45.821892 5826 tablet_service.cc:1467] Processing CreateTablet for tablet 0b96fcd7da1c4e52bcad3779a23e141c (DEFAULT_TABLE table=default.Table_432ac67683bd4fb4a2db65e5369a98c2 [id=7cf1c563f7f74ddaa65c0f335b1a3579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.825821 5603 tablet_service.cc:1467] Processing CreateTablet for tablet 521ffd66d22f48188635b85c6e72f650 (DEFAULT_TABLE table=default.Table_f80c686d970c4c32966a483d401a1e52 [id=0d317758cc2d44d689558f770dae2680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.819108 8141 raft_consensus.cc:482] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:45.829463 8141 raft_consensus.cc:504] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:45.833359 8141 leader_election.cc:288] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:45.796701 4837 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 487) took 2917 ms. Trace:
W20260808 13:35:45.837074 4840 rpcz_store.cc:269] 0808 13:35:44.657129 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.789580 (+1132451us) service_pool.cc:214] Skipping call since client already timed out
0808 13:35:45.789711 (+ 131us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260808 13:35:45.837200 4837 rpcz_store.cc:276] 0808 13:35:42.879089 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:44.047584 (+1168495us) service_pool.cc:227] Handling call
0808 13:35:44.090178 (+ 42594us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:44.092118 (+ 1940us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:44.092724 (+ 606us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:44.092970 (+ 246us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:44.208853 (+115883us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:45.796645 (+1587792us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:44.094229 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:44.094563 (+ 334us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:44.094586 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:44.094597 (+ 11us) tablet.cc:660] Decoding operations
0808 13:35:44.094872 (+ 275us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:44.094912 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:44.094933 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:44.095076 (+ 143us) tablet.cc:699] Row locks acquired
0808 13:35:44.095089 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:44.095190 (+ 101us) write_op.cc:270] Start()
0808 13:35:44.095279 (+ 89us) write_op.cc:276] Timestamp: P: 1786196144095133 usec, L: 0
0808 13:35:44.095293 (+ 14us) op_driver.cc:331] REPLICATION: starting
0808 13:35:44.095744 (+ 451us) log.cc:842] Serialized 405 byte log entry
0808 13:35:44.104934 (+ 9190us) op_driver.cc:444] REPLICATION: finished
0808 13:35:44.105266 (+ 332us) write_op.cc:301] APPLY: starting
0808 13:35:44.105324 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:44.105410 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:44.105516 (+ 106us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:44.105740 (+ 224us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:44.105885 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:44.105938 (+ 53us) write_op.cc:312] APPLY: finished
0808 13:35:44.106114 (+ 176us) log.cc:842] Serialized 23 byte log entry
0808 13:35:44.106263 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:44.106328 (+ 65us) write_op.cc:454] Released schema lock
0808 13:35:44.106356 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:44.106531 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:44.106555 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14741,"child_traces":[["op",{"apply.queue_time_us":182,"apply.run_cpu_time_us":1464,"apply.run_wall_time_us":1463,"num_ops":1,"prepare.queue_time_us":101,"prepare.run_cpu_time_us":1760,"prepare.run_wall_time_us":1760,"raft.queue_time_us":1048,"raft.run_cpu_time_us":630,"raft.run_wall_time_us":626,"replication_time_us":9567}]]}
I20260808 13:35:45.841195 4837 master_service.cc:946] ReplaceTablet: received request to replace tablet 553a01330cfe4dd38a61860dc943fbbc from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:45.844012 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 0b96fcd7da1c4e52bcad3779a23e141c (DEFAULT_TABLE table=default.Table_432ac67683bd4fb4a2db65e5369a98c2 [id=7cf1c563f7f74ddaa65c0f335b1a3579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.848779 6033 tablet_bootstrap.cc:492] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:45.858943 6033 ts_tablet_manager.cc:1397] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.144s user 0.007s sys 0.008s
I20260808 13:35:45.864032 4837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 553a01330cfe4dd38a61860dc943fbbc
I20260808 13:35:45.864436 6046 raft_consensus.cc:348] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.865093 6046 raft_consensus.cc:374] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:45.865362 6046 raft_consensus.cc:729] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:45.866005 4837 catalog_manager.cc:6565] ReplaceTablet: tablet 553a01330cfe4dd38a61860dc943fbbc deleted and replaced by tablet 8f16ee6c8acb40d994347546fa0fdbe7
I20260808 13:35:45.866465 4837 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60270 (request call id 490) took 2467 ms. Trace:
I20260808 13:35:45.871762 5610 tablet_service.cc:1514] Processing DeleteTablet for tablet 553a01330cfe4dd38a61860dc943fbbc with delete_type TABLET_DATA_DELETED (replaced by tablet 8f16ee6c8acb40d994347546fa0fdbe7) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:45.868675 4837 rpcz_store.cc:276] 0808 13:35:43.398590 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.841018 (+2442428us) service_pool.cc:227] Handling call
0808 13:35:45.866345 (+ 25327us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:45.846594 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:45.847420 (+ 826us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:45.847477 (+ 57us) write_op.cc:435] Acquired schema lock
0808 13:35:45.847513 (+ 36us) tablet.cc:660] Decoding operations
0808 13:35:45.848009 (+ 496us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:45.848072 (+ 63us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:45.848106 (+ 34us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:45.848390 (+ 284us) tablet.cc:699] Row locks acquired
0808 13:35:45.848415 (+ 25us) write_op.cc:260] PREPARE: finished
0808 13:35:45.848640 (+ 225us) write_op.cc:270] Start()
0808 13:35:45.848790 (+ 150us) write_op.cc:276] Timestamp: P: 1786196145848529 usec, L: 0
0808 13:35:45.848811 (+ 21us) op_driver.cc:331] REPLICATION: starting
0808 13:35:45.849803 (+ 992us) log.cc:842] Serialized 622 byte log entry
0808 13:35:45.860482 (+ 10679us) op_driver.cc:444] REPLICATION: finished
0808 13:35:45.860975 (+ 493us) write_op.cc:301] APPLY: starting
0808 13:35:45.861077 (+ 102us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:45.861288 (+ 211us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:45.861461 (+ 173us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:45.862124 (+ 663us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:45.862528 (+ 404us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:45.862633 (+ 105us) write_op.cc:312] APPLY: finished
0808 13:35:45.862889 (+ 256us) log.cc:842] Serialized 29 byte log entry
0808 13:35:45.863130 (+ 241us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:45.863256 (+ 126us) write_op.cc:454] Released schema lock
0808 13:35:45.863314 (+ 58us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:45.863589 (+ 275us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:45.863637 (+ 48us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":241,"apply.run_cpu_time_us":2935,"apply.run_wall_time_us":2935,"num_ops":2,"prepare.queue_time_us":120,"prepare.run_cpu_time_us":3648,"prepare.run_wall_time_us":4906,"raft.queue_time_us":136,"raft.run_cpu_time_us":1031,"raft.run_wall_time_us":1031,"replication_time_us":11593}]]}
I20260808 13:35:45.868005 8338 raft_consensus.cc:482] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:45.865985 6046 consensus_queue.cc:260] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.869781 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 6278446e587b42168d388acdf7ec284e (DEFAULT_TABLE table=default.Table_d99de9b6e1e64b48873233ac5849ea09 [id=88fdd9e2283a4018ad2e1d02dc9cd4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.884866 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6278446e587b42168d388acdf7ec284e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.870561 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b57966a9b7b4673ab9f98720cfd456d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:17 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:45.870754 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 369a942248bd4cec9ee9170fe55601a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:45.871232 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:22 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:45.872987 5357 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b07b7d31ef41e484389c0f007598f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:40 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:45.874300 5353 tablet_service.cc:1514] Processing DeleteTablet for tablet daa59a18ae624bc8975bee2c460895a7 with delete_type TABLET_DATA_DELETED (replaced by tablet cb7f6487e28f4aaa94d448be71f8a3c2) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:45.874222 5334 tablet_service.cc:1467] Processing CreateTablet for tablet 521ffd66d22f48188635b85c6e72f650 (DEFAULT_TABLE table=default.Table_f80c686d970c4c32966a483d401a1e52 [id=0d317758cc2d44d689558f770dae2680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.889176 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 521ffd66d22f48188635b85c6e72f650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.876031 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 0b96fcd7da1c4e52bcad3779a23e141c (DEFAULT_TABLE table=default.Table_432ac67683bd4fb4a2db65e5369a98c2 [id=7cf1c563f7f74ddaa65c0f335b1a3579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.875054 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet 553a01330cfe4dd38a61860dc943fbbc with delete_type TABLET_DATA_DELETED (replaced by tablet 8f16ee6c8acb40d994347546fa0fdbe7) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:45.887459 5829 tablet_service.cc:1467] Processing CreateTablet for tablet b973f2094a4e420095757820bbb8a767 (DEFAULT_TABLE table=default.Table_187b31571225409e855d0b4783e4b74c [id=0c03629a169f469897599b731ca031e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.893536 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 521ffd66d22f48188635b85c6e72f650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.898391 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cd8f12cda94c4eb9a848f002a4020b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:37 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:45.912745 5610 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c2b34c95244472beec43643e60ff5f with delete_type TABLET_DATA_DELETED (replaced by tablet b314734698ed423ba9cf4134db5846ec) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:45.914050 5610 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:31 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:45.916232 5610 tablet_service.cc:1467] Processing CreateTablet for tablet b973f2094a4e420095757820bbb8a767 (DEFAULT_TABLE table=default.Table_187b31571225409e855d0b4783e4b74c [id=0c03629a169f469897599b731ca031e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.920020 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b973f2094a4e420095757820bbb8a767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.920050 6026 tablet_bootstrap.cc:492] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:45.920496 6026 ts_tablet_manager.cc:1397] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.272s user 0.011s sys 0.009s
I20260808 13:35:45.922140 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6800eceb96475284fdbffb4d23e5e3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:45.923226 5826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b96fcd7da1c4e52bcad3779a23e141c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.923457 5616 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf47d3df0ac480bb753d6dc46b4d7db (DEFAULT_TABLE table=default.Table_a446b3e7bb7944b0a5b119dd6a1b6d7c [id=4bfd4a2482364643a6d15f7a638f2eaa]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:45.923857 5550 leader_election.cc:341] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:45.931387 5827 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf47d3df0ac480bb753d6dc46b4d7db (DEFAULT_TABLE table=default.Table_a446b3e7bb7944b0a5b119dd6a1b6d7c [id=4bfd4a2482364643a6d15f7a638f2eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.932188 6026 raft_consensus.cc:348] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:45.933117 6026 raft_consensus.cc:374] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:45.933256 5827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf47d3df0ac480bb753d6dc46b4d7db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.933386 6026 raft_consensus.cc:729] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:45.937472 6026 consensus_queue.cc:260] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:45.942000 6026 ts_tablet_manager.cc:1428] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
W20260808 13:35:45.674417 4838 rpcz_store.cc:269] 0808 13:35:41.637947 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.638077 (+ 130us) service_pool.cc:227] Handling call
0808 13:35:45.672246 (+4034169us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:43.429094 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:43.429440 (+ 346us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:43.429462 (+ 22us) write_op.cc:435] Acquired schema lock
0808 13:35:43.429475 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:43.429750 (+ 275us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:43.429783 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:43.429805 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:43.429940 (+ 135us) tablet.cc:699] Row locks acquired
0808 13:35:43.429952 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:43.430047 (+ 95us) write_op.cc:270] Start()
0808 13:35:43.430116 (+ 69us) write_op.cc:276] Timestamp: P: 1786196143429916 usec, L: 0
0808 13:35:43.430131 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:43.430596 (+ 465us) log.cc:842] Serialized 527 byte log entry
0808 13:35:43.447650 (+ 17054us) op_driver.cc:444] REPLICATION: finished
0808 13:35:43.471459 (+ 23809us) write_op.cc:301] APPLY: starting
0808 13:35:43.471531 (+ 72us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:43.471624 (+ 93us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:43.471722 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:43.471918 (+ 196us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:43.472044 (+ 126us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:43.472098 (+ 54us) write_op.cc:312] APPLY: finished
0808 13:35:43.472272 (+ 174us) log.cc:842] Serialized 23 byte log entry
0808 13:35:43.472412 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:43.472478 (+ 66us) write_op.cc:454] Released schema lock
0808 13:35:43.472511 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:43.472728 (+ 217us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:43.472780 (+ 52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41915,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":18858,"apply.run_cpu_time_us":1578,"apply.run_wall_time_us":6326,"num_ops":1,"prepare.queue_time_us":6181,"prepare.run_cpu_time_us":1783,"prepare.run_wall_time_us":4675,"raft.queue_time_us":3101,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":11,"replication_time_us":17456,"thread_start_us":64387,"threads_started":1}]]}
I20260808 13:35:45.951144 5592 tablet_service.cc:1467] Processing CreateTablet for tablet ba6cad4fd8a04f7b87d8e873b8ec8a3f (DEFAULT_TABLE table=default.Table_6b4561dcd8e64216a27618e3599c13d8 [id=fce834aa5aec4694845b2ffc2c9fe962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:45.953243 4838 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_f985e4cebca740fea14f259fcfacec86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:45.835583 4839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 46) took 2438 ms (client timeout 1000 ms). Trace:
W20260808 13:35:45.955184 4839 rpcz_store.cc:269] 0808 13:35:43.397411 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:43.468003 (+ 70592us) service_pool.cc:227] Handling call
0808 13:35:43.513325 (+ 45322us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:45.835508 (+2322183us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:44.206866 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:44.207215 (+ 349us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:44.207238 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:44.207250 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:44.207528 (+ 278us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:44.207567 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:44.207587 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:44.207722 (+ 135us) tablet.cc:699] Row locks acquired
0808 13:35:44.207735 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:44.207840 (+ 105us) write_op.cc:270] Start()
0808 13:35:44.207920 (+ 80us) write_op.cc:276] Timestamp: P: 1786196144207767 usec, L: 0
0808 13:35:44.207935 (+ 15us) op_driver.cc:331] REPLICATION: starting
0808 13:35:44.208421 (+ 486us) log.cc:842] Serialized 356 byte log entry
0808 13:35:44.224853 (+ 16432us) op_driver.cc:444] REPLICATION: finished
0808 13:35:44.225145 (+ 292us) write_op.cc:301] APPLY: starting
0808 13:35:44.225208 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:44.225307 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:44.225412 (+ 105us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:44.225603 (+ 191us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:44.225740 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:44.225783 (+ 43us) write_op.cc:312] APPLY: finished
0808 13:35:44.225967 (+ 184us) log.cc:842] Serialized 23 byte log entry
0808 13:35:44.226113 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:44.226173 (+ 60us) write_op.cc:454] Released schema lock
0808 13:35:44.226205 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:44.226375 (+ 170us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:44.226398 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":212,"reported_tablets":29,"spinlock_wait_cycles":83992960,"child_traces":[["op",{"apply.queue_time_us":153,"apply.run_cpu_time_us":1399,"apply.run_wall_time_us":1399,"num_ops":1,"prepare.queue_time_us":94,"prepare.run_cpu_time_us":1821,"prepare.run_wall_time_us":1820,"raft.queue_time_us":915,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":547,"replication_time_us":16843}]]}
W20260808 13:35:45.808389 4833 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 61) took 2178 ms (client timeout 1000 ms). Trace:
W20260808 13:35:45.959122 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 55) took 1753 ms (client timeout 1000 ms). Trace:
I20260808 13:35:45.963450 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 521ffd66d22f48188635b85c6e72f650. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:45.779771 4835 rpcz_store.cc:269] 0808 13:35:41.265759 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:41.265872 (+ 113us) service_pool.cc:227] Handling call
0808 13:35:41.315043 (+ 49171us) spinlock_profiling.cc:243] Waited 45.4 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:45.771816 (+4456773us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:43.900149 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:43.900506 (+ 357us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:43.900529 (+ 23us) write_op.cc:435] Acquired schema lock
0808 13:35:43.900542 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:43.900831 (+ 289us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:43.900871 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:43.900891 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:43.901031 (+ 140us) tablet.cc:699] Row locks acquired
0808 13:35:43.901043 (+ 12us) write_op.cc:260] PREPARE: finished
0808 13:35:43.901146 (+ 103us) write_op.cc:270] Start()
0808 13:35:43.901225 (+ 79us) write_op.cc:276] Timestamp: P: 1786196143900962 usec, L: 0
0808 13:35:43.901241 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:43.901715 (+ 474us) log.cc:842] Serialized 356 byte log entry
0808 13:35:43.924986 (+ 23271us) op_driver.cc:444] REPLICATION: finished
0808 13:35:43.925342 (+ 356us) write_op.cc:301] APPLY: starting
0808 13:35:43.925394 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:43.925484 (+ 90us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:43.925597 (+ 113us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:43.925837 (+ 240us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:43.925983 (+ 146us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:43.926026 (+ 43us) write_op.cc:312] APPLY: finished
0808 13:35:43.926226 (+ 200us) log.cc:842] Serialized 23 byte log entry
0808 13:35:43.926398 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:43.926462 (+ 64us) write_op.cc:454] Released schema lock
0808 13:35:43.926491 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:43.926678 (+ 187us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:43.926700 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":15,"spinlock_wait_cycles":104367488,"child_traces":[["op",{"apply.queue_time_us":177,"apply.run_cpu_time_us":1525,"apply.run_wall_time_us":1525,"num_ops":1,"prepare.queue_time_us":109,"prepare.run_cpu_time_us":1822,"prepare.run_wall_time_us":1822,"raft.queue_time_us":1288,"raft.run_cpu_time_us":553,"raft.run_wall_time_us":556,"replication_time_us":23674}]]}
I20260808 13:35:45.966200 4835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_c171ae1c3b234b4e8b6f3def21f27924"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:45.967125 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b96fcd7da1c4e52bcad3779a23e141c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:45.836640 4841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 60) took 3291 ms (client timeout 1000 ms). Trace:
W20260808 13:35:45.967967 4841 rpcz_store.cc:269] 0808 13:35:42.544769 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:42.669957 (+125188us) service_pool.cc:227] Handling call
0808 13:35:45.836123 (+3166166us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b2400003008. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227511ef11 00007f22752a2851 00007f22752a2702 00007f227e139b09 00007f2275ccb480 00007f2275cd83cf 00007f227e1fe6d4 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec
0808 13:35:45.836572 (+ 449us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4408,"reported_tablets":17,"spinlock_wait_cycles":2343296}
I20260808 13:35:45.970321 4841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_efd769b05e1142088d56528e943072ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:45.972023 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b973f2094a4e420095757820bbb8a767. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:45.974306 4835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c171ae1c3b234b4e8b6f3def21f27924 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:35:45.976886 4841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efd769b05e1142088d56528e943072ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:45.978578 4839 master_service.cc:946] ReplaceTablet: received request to replace tablet 16736500f94c43d3a3163f61b7606aad from {username='slave'} at 127.0.0.1:60270
W20260808 13:35:45.958779 4833 rpcz_store.cc:269] 0808 13:35:43.630209 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:43.835308 (+205099us) service_pool.cc:227] Handling call
0808 13:35:45.340731 (+1505423us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:45.340758 (+ 27us) write_op.cc:454] Released schema lock
0808 13:35:45.808319 (+467561us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:45.260890 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:45.261797 (+ 907us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:45.261850 (+ 53us) write_op.cc:435] Acquired schema lock
0808 13:35:45.261898 (+ 48us) tablet.cc:660] Decoding operations
0808 13:35:45.262492 (+ 594us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:45.262587 (+ 95us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:45.262864 (+ 277us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:45.263223 (+ 359us) tablet.cc:699] Row locks acquired
0808 13:35:45.263268 (+ 45us) write_op.cc:260] PREPARE: finished
0808 13:35:45.263539 (+ 271us) write_op.cc:270] Start()
0808 13:35:45.263728 (+ 189us) write_op.cc:276] Timestamp: P: 1786196145263433 usec, L: 0
0808 13:35:45.263772 (+ 44us) op_driver.cc:331] REPLICATION: starting
0808 13:35:45.264754 (+ 982us) log.cc:842] Serialized 527 byte log entry
0808 13:35:45.274667 (+ 9913us) op_driver.cc:444] REPLICATION: finished
0808 13:35:45.320256 (+ 45589us) write_op.cc:301] APPLY: starting
0808 13:35:45.320316 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:45.320410 (+ 94us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:45.320502 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:45.320695 (+ 193us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:45.320832 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:45.320873 (+ 41us) write_op.cc:312] APPLY: finished
0808 13:35:45.321042 (+ 169us) log.cc:842] Serialized 23 byte log entry
0808 13:35:45.321207 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:45.321276 (+ 69us) write_op.cc:454] Released schema lock
0808 13:35:45.321305 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":10635,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":382,"apply.run_cpu_time_us":1274,"apply.run_wall_time_us":1274,"num_ops":1,"prepare.queue_time_us":94,"prepare.run_cpu_time_us":4435,"prepare.run_wall_time_us":6167,"raft.queue_time_us":182,"raft.run_cpu_time_us":1283,"raft.run_wall_time_us":1293,"replication_time_us":10792}]]}
I20260808 13:35:45.979416 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf47d3df0ac480bb753d6dc46b4d7db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.979629 6046 ts_tablet_manager.cc:1428] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.189s user 0.006s sys 0.001s
I20260808 13:35:45.980623 6046 tablet_bootstrap.cc:492] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:45.981063 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b96fcd7da1c4e52bcad3779a23e141c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.980985 4833 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_4ce2d44dea88418eb0e636202d9df995" } new_table_name: "default.Table_d2c33a8a16d34fb0a22e445a3c1be52a" modify_external_catalogs: true
I20260808 13:35:45.883358 8338 raft_consensus.cc:504] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:45.985800 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6cad4fd8a04f7b87d8e873b8ec8a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:45.987164 6026 tablet_bootstrap.cc:492] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:45.997905 4838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f985e4cebca740fea14f259fcfacec86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:46.011020 5338 tablet_service.cc:1514] Processing DeleteTablet for tablet 553a01330cfe4dd38a61860dc943fbbc with delete_type TABLET_DATA_DELETED (replaced by tablet 8f16ee6c8acb40d994347546fa0fdbe7) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:46.030503 5831 tablet_service.cc:1467] Processing CreateTablet for tablet b3d489bfe6724b74a830217bfc45a84a (DEFAULT_TABLE table=default.Table_7700dbef158a4ddfa8a3122c18e38817 [id=e6381ed760224b53be85a88376ce8962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.034353 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 0db761bf06ae47fe8ab1eea6de408e6f (DEFAULT_TABLE table=default.Table_efea7e3ddc70498f94c05ddee5ba3d76 [id=c965816b949348ba98967bdc0115eefa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.040498 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db761bf06ae47fe8ab1eea6de408e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.044569 5831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d489bfe6724b74a830217bfc45a84a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:45.959427 4832 rpcz_store.cc:269] 0808 13:35:44.205679 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:44.215631 (+ 9952us) service_pool.cc:227] Handling call
0808 13:35:45.512412 (+1296781us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:45.512447 (+ 35us) write_op.cc:454] Released schema lock
0808 13:35:45.959056 (+446609us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:45.454664 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:45.455493 (+ 829us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:45.455559 (+ 66us) write_op.cc:435] Acquired schema lock
0808 13:35:45.455571 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:45.455972 (+ 401us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:45.456033 (+ 61us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:45.456086 (+ 53us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:45.456284 (+ 198us) tablet.cc:699] Row locks acquired
0808 13:35:45.456307 (+ 23us) write_op.cc:260] PREPARE: finished
0808 13:35:45.456568 (+ 261us) write_op.cc:270] Start()
0808 13:35:45.456703 (+ 135us) write_op.cc:276] Timestamp: P: 1786196145456469 usec, L: 0
0808 13:35:45.456719 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:45.457612 (+ 893us) log.cc:842] Serialized 527 byte log entry
0808 13:35:45.477002 (+ 19390us) op_driver.cc:444] REPLICATION: finished
0808 13:35:45.477952 (+ 950us) write_op.cc:301] APPLY: starting
0808 13:35:45.478088 (+ 136us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:45.478267 (+ 179us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:45.478452 (+ 185us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:45.478840 (+ 388us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:45.479050 (+ 210us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:45.479121 (+ 71us) write_op.cc:312] APPLY: finished
0808 13:35:45.479334 (+ 213us) log.cc:842] Serialized 23 byte log entry
0808 13:35:45.479576 (+ 242us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:45.479683 (+ 107us) write_op.cc:454] Released schema lock
0808 13:35:45.479741 (+ 58us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0808 13:35:45.633739 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:45.634214 (+ 475us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:45.634241 (+ 27us) write_op.cc:435] Acquired schema lock
0808 13:35:45.634257 (+ 16us) tablet.cc:660] Decoding operations
0808 13:35:45.634570 (+ 313us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:45.634637 (+ 67us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:45.634662 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:45.634831 (+ 169us) tablet.cc:699] Row locks acquired
0808 13:35:45.634848 (+ 17us) write_op.cc:260] PREPARE: finished
0808 13:35:45.634974 (+ 126us) write_op.cc:270] Start()
0808 13:35:45.635062 (+ 88us) write_op.cc:276] Timestamp: P: 1786196145634920 usec, L: 0
0808 13:35:45.635080 (+ 18us) op_driver.cc:331] REPLICATION: starting
0808 13:35:45.635674 (+ 594us) log.cc:842] Serialized 356 byte log entry
0808 13:35:45.645757 (+ 10083us) op_driver.cc:444] REPLICATION: finished
0808 13:35:45.646258 (+ 501us) write_op.cc:301] APPLY: starting
0808 13:35:45.646317 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:45.646475 (+ 158us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:45.646598 (+ 123us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:45.646914 (+ 316us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:45.647209 (+ 295us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:45.647275 (+ 66us) write_op.cc:312] APPLY: finished
0808 13:35:45.647513 (+ 238us) log.cc:842] Serialized 23 byte log entry
0808 13:35:45.647697 (+ 184us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:45.647805 (+ 108us) write_op.cc:454] Released schema lock
0808 13:35:45.647848 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:45.648154 (+ 306us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:45.648209 (+ 55us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":87445,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":662,"apply.run_cpu_time_us":2132,"apply.run_wall_time_us":2132,"num_ops":1,"prepare.queue_time_us":116,"prepare.run_cpu_time_us":3344,"prepare.run_wall_time_us":10551,"raft.queue_time_us":95,"raft.run_cpu_time_us":1196,"raft.run_wall_time_us":1196,"replication_time_us":20111}],["op",{"apply.queue_time_us":254,"apply.run_cpu_time_us":2151,"apply.run_wall_time_us":2150,"num_ops":1,"prepare.queue_time_us":111056,"prepare.run_cpu_time_us":2291,"prepare.run_wall_time_us":2292,"raft.queue_time_us":220,"raft.run_cpu_time_us":670,"raft.run_wall_time_us":669,"replication_time_us":10577}]]}
I20260808 13:35:46.061719 5614 tablet_service.cc:1467] Processing CreateTablet for tablet b3d489bfe6724b74a830217bfc45a84a (DEFAULT_TABLE table=default.Table_7700dbef158a4ddfa8a3122c18e38817 [id=e6381ed760224b53be85a88376ce8962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.062940 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d489bfe6724b74a830217bfc45a84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.064149 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:46.064833 5630 raft_consensus.cc:2454] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:46.066133 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 0db761bf06ae47fe8ab1eea6de408e6f (DEFAULT_TABLE table=default.Table_efea7e3ddc70498f94c05ddee5ba3d76 [id=c965816b949348ba98967bdc0115eefa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.067288 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db761bf06ae47fe8ab1eea6de408e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.073419 5285 leader_election.cc:302] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:46.076833 8338 leader_election.cc:288] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:46.079267 6046 tablet_bootstrap.cc:654] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:46.109463 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 8b90b066ebf3433fb0df9dd90865bc62 (DEFAULT_TABLE table=default.Table_5db9de8011f74b9f9192b39113dfa0c8 [id=dfd09716123a4de7b9fcdd65d1b9ef61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.110365 5598 tablet_service.cc:1467] Processing CreateTablet for tablet ccef7d478a31444481546a15a95e02d9 (DEFAULT_TABLE table=default.Table_9936d69a1f7b4853a7c37d5e8da2077f [id=a7e1be7320c34f3bab091c92ec2c1f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.115713 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 54cd7746bb6a4252b74517999782bbab (DEFAULT_TABLE table=default.Table_0f52af74f5c34a0aa2f84a521c188673 [id=2378817b60944a789ff39c84ebdc8991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.117199 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_9d2535df26964377be89e6da0a54e8b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:35:46.119341 6026 tablet_bootstrap.cc:654] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:46.119807 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d2535df26964377be89e6da0a54e8b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:46.155568 5843 tablet_service.cc:1467] Processing CreateTablet for tablet ba6cad4fd8a04f7b87d8e873b8ec8a3f (DEFAULT_TABLE table=default.Table_6b4561dcd8e64216a27618e3599c13d8 [id=fce834aa5aec4694845b2ffc2c9fe962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.157053 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6cad4fd8a04f7b87d8e873b8ec8a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.163448 4838 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 491) took 2755 ms. Trace:
I20260808 13:35:46.170178 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b90b066ebf3433fb0df9dd90865bc62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.175469 5598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccef7d478a31444481546a15a95e02d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.176913 8306 ts_tablet_manager.cc:1918] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:46.177690 8306 log.cc:1196] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/3a27517560d64a3b8da2cf55980e18a8
I20260808 13:35:46.179271 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54cd7746bb6a4252b74517999782bbab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.180555 8357 raft_consensus.cc:482] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:46.181321 8357 raft_consensus.cc:504] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.185478 8357 leader_election.cc:288] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:46.191560 6026 tablet_bootstrap.cc:492] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:46.191968 6026 ts_tablet_manager.cc:1397] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.205s user 0.011s sys 0.006s
I20260808 13:35:46.208061 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 54cd7746bb6a4252b74517999782bbab (DEFAULT_TABLE table=default.Table_0f52af74f5c34a0aa2f84a521c188673 [id=2378817b60944a789ff39c84ebdc8991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.209797 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54cd7746bb6a4252b74517999782bbab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:46.210834 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 523630us
I20260808 13:35:46.219555 8315 consensus_queue.cc:1035] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260808 13:35:46.235857 5842 tablet_service.cc:1467] Processing CreateTablet for tablet ccef7d478a31444481546a15a95e02d9 (DEFAULT_TABLE table=default.Table_9936d69a1f7b4853a7c37d5e8da2077f [id=a7e1be7320c34f3bab091c92ec2c1f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.237155 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccef7d478a31444481546a15a95e02d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.163839 4838 rpcz_store.cc:276] 0808 13:35:43.408298 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.952741 (+2544443us) service_pool.cc:227] Handling call
0808 13:35:45.997697 (+ 44956us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b4000001378. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227511ef11 00007f22752a3216 00007f227e27563e 00007f227e2755b8 00007f227e1238b1 00007f227e0dcb31 00007f227e1ff18f 00007f227bd3c034 00007f227bd3bfc2 00007f227bd3bef1 00007f227bd3be6c
0808 13:35:46.054162 (+ 56465us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:46.055703 (+ 1541us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:46.162773 (+107070us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:46.163070 (+ 297us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:46.163385 (+ 315us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:46.137234 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:46.138086 (+ 852us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:46.138140 (+ 54us) write_op.cc:435] Acquired schema lock
0808 13:35:46.138182 (+ 42us) tablet.cc:660] Decoding operations
0808 13:35:46.138916 (+ 734us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:46.138985 (+ 69us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:46.139040 (+ 55us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:46.139507 (+ 467us) tablet.cc:699] Row locks acquired
0808 13:35:46.139549 (+ 42us) write_op.cc:260] PREPARE: finished
0808 13:35:46.139775 (+ 226us) write_op.cc:270] Start()
0808 13:35:46.139944 (+ 169us) write_op.cc:276] Timestamp: P: 1786196146139688 usec, L: 0
0808 13:35:46.140007 (+ 63us) op_driver.cc:331] REPLICATION: starting
0808 13:35:46.141085 (+ 1078us) log.cc:842] Serialized 433 byte log entry
0808 13:35:46.153185 (+ 12100us) op_driver.cc:444] REPLICATION: finished
0808 13:35:46.153636 (+ 451us) write_op.cc:301] APPLY: starting
0808 13:35:46.153705 (+ 69us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:46.153847 (+ 142us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:46.153969 (+ 122us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:46.154601 (+ 632us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:46.154826 (+ 225us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:46.154911 (+ 85us) write_op.cc:312] APPLY: finished
0808 13:35:46.155196 (+ 285us) log.cc:842] Serialized 29 byte log entry
0808 13:35:46.155336 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:46.155434 (+ 98us) write_op.cc:454] Released schema lock
0808 13:35:46.155488 (+ 54us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:46.155689 (+ 201us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:46.155730 (+ 41us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":8040704,"child_traces":[["op",{"apply.queue_time_us":235,"apply.run_cpu_time_us":2314,"apply.run_wall_time_us":2321,"mutex_wait_us":79962,"num_ops":2,"prepare.queue_time_us":80119,"prepare.run_cpu_time_us":4405,"prepare.run_wall_time_us":12427,"raft.queue_time_us":158,"raft.run_cpu_time_us":1106,"raft.run_wall_time_us":1262,"replication_time_us":13054}]]}
I20260808 13:35:46.240072 4838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_00b62ebc438b40209bd662e8cd04fb29" } modify_external_catalogs: true
I20260808 13:35:46.256318 8306 ts_tablet_manager.cc:1939] T 3a27517560d64a3b8da2cf55980e18a8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:46.168251 8336 raft_consensus.cc:482] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:46.258896 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba39706cff65404485959865fff1772a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:46.263447 6026 raft_consensus.cc:348] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.255709 8048 consensus_queue.cc:1035] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260808 13:35:46.260319 8336 raft_consensus.cc:504] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.280262 8340 raft_consensus.cc:2781] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:35:46.286051 8340 raft_consensus.cc:482] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 98dd96f425fc4ab3a6a54f2dfaaf9fab)
I20260808 13:35:46.286454 8340 raft_consensus.cc:3037] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:46.283310 8338 consensus_queue.cc:1035] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260808 13:35:46.295411 8336 leader_election.cc:288] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:46.306675 5009 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9ca0 after lost signal to thread 4913
W20260808 13:35:46.307426 5009 debug-util.cc:398] Leaking SignalData structure 0x7b08000b6da0 after lost signal to thread 5014
I20260808 13:35:46.296835 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249dcaf769934b75945d79d827a96be6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:46.312891 6026 raft_consensus.cc:374] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:46.313149 5629 raft_consensus.cc:2454] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:46.313165 6026 raft_consensus.cc:729] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:46.316262 5285 leader_election.cc:302] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:46.312506 8306 tablet_replica.cc:330] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:46.314216 6026 consensus_queue.cc:260] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.326424 8306 raft_consensus.cc:2229] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:46.328114 8306 pending_rounds.cc:62] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Trying to abort 1 pending ops.
I20260808 13:35:46.328420 8306 pending_rounds.cc:68] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259407533748224 op_type: NO_OP noop_request { }
I20260808 13:35:46.326386 8338 raft_consensus.cc:2781] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:46.329731 8338 raft_consensus.cc:482] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:46.330065 8338 raft_consensus.cc:3037] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:46.327679 6026 ts_tablet_manager.cc:1428] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.135s user 0.006s sys 0.002s
I20260808 13:35:46.330209 8306 raft_consensus.cc:2866] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:35:46.333730 8306 raft_consensus.cc:2258] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:46.335515 6026 tablet_bootstrap.cc:492] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:46.339273 8356 raft_consensus.cc:482] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:46.339697 8356 raft_consensus.cc:504] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.367600 8356 leader_election.cc:288] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:46.370968 6026 tablet_bootstrap.cc:654] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:46.379047 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6800eceb96475284fdbffb4d23e5e3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:46.390594 8363 tablet_replica.cc:330] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:46.391558 8363 raft_consensus.cc:2229] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:46.436971 8306 ts_tablet_manager.cc:1905] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:46.400063 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff163c36c6d42238cce2396102f9a68" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:46.392102 8363 raft_consensus.cc:2258] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:46.402006 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 145672us
I20260808 13:35:46.437912 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594afd4174644c218951b9df82360bfe" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:46.315589 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 3a27517560d64a3b8da2cf55980e18a8 (table default.Table_0a3697cdad3740a1baa47b3075b5e1f9 [id=19a3a9ca647c4e069a20960c7f4cc57a]) successfully deleted
W20260808 13:35:46.479259 4821 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 163696us
W20260808 13:35:46.479907 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 7e8cf06462484c99a860762a82f7ad64 because the tablet was not found. No further retry: Not found: Tablet not found: 7e8cf06462484c99a860762a82f7ad64
I20260808 13:35:46.482517 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae01fdfbc564fb8abf9b6bf9a308837 (DEFAULT_TABLE table=default.Table_025eb6ef4e024bec917c3272ba1b8c35 [id=7dda6de1004344a987b5ede426b5a758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.266484 6033 raft_consensus.cc:348] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.498809 6033 raft_consensus.cc:374] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260808 13:35:46.364310 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 44096.6us
I20260808 13:35:46.374697 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 8b90b066ebf3433fb0df9dd90865bc62 (DEFAULT_TABLE table=default.Table_5db9de8011f74b9f9192b39113dfa0c8 [id=dfd09716123a4de7b9fcdd65d1b9ef61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.382068 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:46.498442 4838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_00b62ebc438b40209bd662e8cd04fb29" } modify_external_catalogs: true
I20260808 13:35:46.420262 8340 raft_consensus.cc:504] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.383030 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff163c36c6d42238cce2396102f9a68" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:46.476823 5838 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae01fdfbc564fb8abf9b6bf9a308837 (DEFAULT_TABLE table=default.Table_025eb6ef4e024bec917c3272ba1b8c35 [id=7dda6de1004344a987b5ede426b5a758]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:35:46.508986 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2d8e4af746774747a70d425068e8dd63 (table default.Table_938ca078ec5e4411b4be33056272983c [id=95f59bd3447c44c291270c448f53d78b], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2d8e4af746774747a70d425068e8dd63 consensus state has no leader
I20260808 13:35:46.499127 6033 raft_consensus.cc:729] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:46.466430 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
I20260808 13:35:46.513710 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:46.515108 5838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae01fdfbc564fb8abf9b6bf9a308837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.515686 6026 tablet_bootstrap.cc:492] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:46.515805 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae01fdfbc564fb8abf9b6bf9a308837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.516119 6026 ts_tablet_manager.cc:1397] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.181s user 0.007s sys 0.006s
W20260808 13:35:46.516688 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 88432.6us
I20260808 13:35:46.516587 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 932a328069bd4edf8137f49a5da66a22 (DEFAULT_TABLE table=default.Table_0fdee57de12549eab898e92f30fe691d [id=9ad354d775a84d32bd5eb627356e0a6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.517712 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 932a328069bd4edf8137f49a5da66a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.520601 6033 consensus_queue.cc:260] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.527143 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b90b066ebf3433fb0df9dd90865bc62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.555708 8340 leader_election.cc:288] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:46.555943 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249dcaf769934b75945d79d827a96be6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:46.556859 5880 raft_consensus.cc:2454] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
W20260808 13:35:46.560465 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 8b57966a9b7b4673ab9f98720cfd456d because the tablet was not found. No further retry: Not found: Tablet not found: 8b57966a9b7b4673ab9f98720cfd456d
I20260808 13:35:46.561360 6033 ts_tablet_manager.cc:1428] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.654s user 0.007s sys 0.001s
I20260808 13:35:46.467581 4835 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 492) took 3058 ms. Trace:
W20260808 13:35:46.561609 6033 ts_tablet_manager.cc:1460] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Tablet startup took 654ms
I20260808 13:35:46.588419 5592 tablet_service.cc:1467] Processing CreateTablet for tablet 932a328069bd4edf8137f49a5da66a22 (DEFAULT_TABLE table=default.Table_0fdee57de12549eab898e92f30fe691d [id=9ad354d775a84d32bd5eb627356e0a6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.586421 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:46.530764 8338 raft_consensus.cc:504] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:46.592548 5550 leader_election.cc:341] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:46.596182 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:46.596863 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447]). Expected version 1 got 0
I20260808 13:35:46.597705 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 52deb2e968c04b99a436e04ae78d9969 (table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445]). Expected version 2 got 0
I20260808 13:35:46.537003 6046 tablet_bootstrap.cc:492] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:46.540125 8237 raft_consensus.cc:482] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:46.598268 8237 raft_consensus.cc:504] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:46.600164 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249dcaf769934b75945d79d827a96be6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:46.600926 5644 raft_consensus.cc:3037] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:46.602221 8237 leader_election.cc:288] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:46.603176 6046 ts_tablet_manager.cc:1397] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.623s user 0.013s sys 0.003s
I20260808 13:35:46.605837 5613 tablet_service.cc:1467] Processing CreateTablet for tablet 603ab05a512d4ed1bcac2a5b86d6ee7d (DEFAULT_TABLE table=default.Table_0e051b613c6e452f89c46cb5aff7f7dd [id=5b28fc73db9a4600bfa8c43e9929b4cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.590556 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 932a328069bd4edf8137f49a5da66a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.614971 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 603ab05a512d4ed1bcac2a5b86d6ee7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.626506 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362dbd6990e740b39690537af98aa2f8" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:46.632967 5285 leader_election.cc:341] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Illegal state: Tablet not RUNNING: STOPPED
I20260808 13:35:46.534307 8363 ts_tablet_manager.cc:1905] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:46.644656 8338 leader_election.cc:288] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:46.648150 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594afd4174644c218951b9df82360bfe" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:46.637246 5551 leader_election.cc:341] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:46.650853 5551 leader_election.cc:302] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:46.659308 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 67690.8us
W20260808 13:35:46.571491 6033 ts_tablet_manager.cc:1462] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Trace:
0808 13:35:21.597633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:21.597768 (+ 135us) service_pool.cc:227] Handling call
0808 13:35:21.599227 (+ 1459us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:21.599377 (+ 150us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:21.644478 (+ 45101us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:22.207919 (+563441us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:22.208491 (+ 572us) inbound_call.cc:173] Queueing success response
0808 13:35:45.714790 (+23506299us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0808 13:35:45.907423 (+192633us) ts_tablet_manager.cc:1429] Starting tablet replica
0808 13:35:46.207740 (+300317us) tablet_replica.cc:259] Starting instrumentation
0808 13:35:46.208207 (+ 467us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":49390,"raft.queue_time_us":33028,"spinlock_wait_cycles":24324480,"tablet-open.queue_time_us":23506458}
I20260808 13:35:46.664418 6046 raft_consensus.cc:348] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.665198 6046 raft_consensus.cc:374] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:46.665462 6046 raft_consensus.cc:729] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:46.666060 6046 consensus_queue.cc:260] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.669904 5332 tablet_service.cc:1467] Processing CreateTablet for tablet b973f2094a4e420095757820bbb8a767 (DEFAULT_TABLE table=default.Table_187b31571225409e855d0b4783e4b74c [id=0c03629a169f469897599b731ca031e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.671769 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf47d3df0ac480bb753d6dc46b4d7db (DEFAULT_TABLE table=default.Table_a446b3e7bb7944b0a5b119dd6a1b6d7c [id=4bfd4a2482364643a6d15f7a638f2eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.673591 5334 tablet_service.cc:1467] Processing CreateTablet for tablet ba6cad4fd8a04f7b87d8e873b8ec8a3f (DEFAULT_TABLE table=default.Table_6b4561dcd8e64216a27618e3599c13d8 [id=fce834aa5aec4694845b2ffc2c9fe962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.673847 5381 raft_consensus.cc:3037] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:46.689549 8280 catalog_manager.cc:1259] Loaded cluster ID: 9d0f658ea6c34f378fe8b1ad811bf2bf
I20260808 13:35:46.689895 8280 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: loading cluster ID for follower catalog manager: success
I20260808 13:35:46.691362 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b973f2094a4e420095757820bbb8a767. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:46.694664 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 101324us
I20260808 13:35:46.695289 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf47d3df0ac480bb753d6dc46b4d7db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.698984 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6cad4fd8a04f7b87d8e873b8ec8a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.698506 5644 raft_consensus.cc:2454] T 249dcaf769934b75945d79d827a96be6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:46.723441 8356 raft_consensus.cc:2726] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:46.724877 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 87657.5us
I20260808 13:35:46.724514 6026 raft_consensus.cc:348] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:46.725705 6026 raft_consensus.cc:374] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:46.729549 6026 raft_consensus.cc:729] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:46.730367 6026 consensus_queue.cc:260] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:46.741124 6033 tablet_bootstrap.cc:492] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:46.741226 5550 leader_election.cc:341] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:46.742466 8403 raft_consensus.cc:482] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:46.743150 8403 raft_consensus.cc:504] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:46.746686 8403 leader_election.cc:288] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:46.751633 5354 tablet_service.cc:1467] Processing CreateTablet for tablet b3d489bfe6724b74a830217bfc45a84a (DEFAULT_TABLE table=default.Table_7700dbef158a4ddfa8a3122c18e38817 [id=e6381ed760224b53be85a88376ce8962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.753126 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d489bfe6724b74a830217bfc45a84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.733716 6026 ts_tablet_manager.cc:1428] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.217s user 0.005s sys 0.000s
W20260808 13:35:46.727890 5551 leader_election.cc:341] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:46.766788 5551 leader_election.cc:302] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:46.768232 8393 raft_consensus.cc:2726] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:46.768945 5353 tablet_service.cc:1467] Processing CreateTablet for tablet ccef7d478a31444481546a15a95e02d9 (DEFAULT_TABLE table=default.Table_9936d69a1f7b4853a7c37d5e8da2077f [id=a7e1be7320c34f3bab091c92ec2c1f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.769972 5337 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae01fdfbc564fb8abf9b6bf9a308837 (DEFAULT_TABLE table=default.Table_025eb6ef4e024bec917c3272ba1b8c35 [id=7dda6de1004344a987b5ede426b5a758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.770465 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccef7d478a31444481546a15a95e02d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.656924 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 2d8e4af746774747a70d425068e8dd63 (table default.Table_938ca078ec5e4411b4be33056272983c [id=95f59bd3447c44c291270c448f53d78b]). Expected version 2 got 0
I20260808 13:35:46.564740 4835 rpcz_store.cc:276] 0808 13:35:43.408774 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.965520 (+2556746us) service_pool.cc:227] Handling call
0808 13:35:46.053512 (+ 87992us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:46.055189 (+ 1677us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:46.466692 (+411503us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:46.466718 (+ 26us) write_op.cc:454] Released schema lock
0808 13:35:46.466936 (+ 218us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:46.467246 (+ 310us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:46.467513 (+ 267us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:46.105645 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:46.106538 (+ 893us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:46.106589 (+ 51us) write_op.cc:435] Acquired schema lock
0808 13:35:46.106638 (+ 49us) tablet.cc:660] Decoding operations
0808 13:35:46.107223 (+ 585us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:46.107275 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:46.107329 (+ 54us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:46.107784 (+ 455us) tablet.cc:699] Row locks acquired
0808 13:35:46.107834 (+ 50us) write_op.cc:260] PREPARE: finished
0808 13:35:46.108104 (+ 270us) write_op.cc:270] Start()
0808 13:35:46.108256 (+ 152us) write_op.cc:276] Timestamp: P: 1786196146108027 usec, L: 0
0808 13:35:46.108278 (+ 22us) op_driver.cc:331] REPLICATION: starting
0808 13:35:46.111044 (+ 2766us) log.cc:842] Serialized 433 byte log entry
0808 13:35:46.126506 (+ 15462us) op_driver.cc:444] REPLICATION: finished
0808 13:35:46.126914 (+ 408us) write_op.cc:301] APPLY: starting
0808 13:35:46.126972 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:46.127149 (+ 177us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:46.127266 (+ 117us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:46.128182 (+ 916us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:46.128487 (+ 305us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:46.128564 (+ 77us) write_op.cc:312] APPLY: finished
0808 13:35:46.128971 (+ 407us) log.cc:842] Serialized 29 byte log entry
0808 13:35:46.129209 (+ 238us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:46.129328 (+ 119us) write_op.cc:454] Released schema lock
0808 13:35:46.129379 (+ 51us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":190,"apply.run_cpu_time_us":2809,"apply.run_wall_time_us":2808,"num_ops":2,"prepare.queue_time_us":49086,"prepare.run_cpu_time_us":4141,"prepare.run_wall_time_us":18325,"raft.queue_time_us":124,"raft.run_cpu_time_us":1215,"raft.run_wall_time_us":1215,"replication_time_us":18112}]]}
I20260808 13:35:46.786080 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f85b4fd30d6c4903946891a399acf7a0 (table default.Table_f042821be4a8421ea3f64f33a071b22d [id=ea2ba343c17d403d8cb0bf9d80bee453]). Expected version 2 got 0
I20260808 13:35:46.788157 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef8c23840c848ddb7d8b1023a839e94 (DEFAULT_TABLE table=default.Table_52f5a796f61f4be79a196bc23065491b [id=ee175d53df794e6aaf29faff1fc2bb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.788552 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae01fdfbc564fb8abf9b6bf9a308837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.773454 5336 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef8c23840c848ddb7d8b1023a839e94 (DEFAULT_TABLE table=default.Table_52f5a796f61f4be79a196bc23065491b [id=ee175d53df794e6aaf29faff1fc2bb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.794471 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef8c23840c848ddb7d8b1023a839e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.775641 5338 tablet_service.cc:1467] Processing CreateTablet for tablet 8b90b066ebf3433fb0df9dd90865bc62 (DEFAULT_TABLE table=default.Table_5db9de8011f74b9f9192b39113dfa0c8 [id=dfd09716123a4de7b9fcdd65d1b9ef61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.777172 5357 tablet_service.cc:1467] Processing CreateTablet for tablet 54cd7746bb6a4252b74517999782bbab (DEFAULT_TABLE table=default.Table_0f52af74f5c34a0aa2f84a521c188673 [id=2378817b60944a789ff39c84ebdc8991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.781011 5335 tablet_service.cc:1467] Processing CreateTablet for tablet 603ab05a512d4ed1bcac2a5b86d6ee7d (DEFAULT_TABLE table=default.Table_0e051b613c6e452f89c46cb5aff7f7dd [id=5b28fc73db9a4600bfa8c43e9929b4cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.805825 4836 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 96bed5e6d1aa49a9b14ba86403c985ce (table default.Table_db41a83ae3f34453af64e73302451c19 [id=1816314c549642e3adc9f39e532ab1ca]). Expected version 1 got 0
I20260808 13:35:46.806806 6026 tablet_bootstrap.cc:492] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:46.808550 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef8c23840c848ddb7d8b1023a839e94. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:46.816902 5551 leader_election.cc:341] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:46.817391 5551 leader_election.cc:302] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:46.818224 8356 raft_consensus.cc:2726] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:46.819533 5551 leader_election.cc:341] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:46.819887 5551 leader_election.cc:302] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:46.822955 8393 raft_consensus.cc:2726] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:46.824836 8306 ts_tablet_manager.cc:1918] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:46.825519 8306 log.cc:1196] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/362dbd6990e740b39690537af98aa2f8
I20260808 13:35:46.826839 8306 ts_tablet_manager.cc:1939] T 362dbd6990e740b39690537af98aa2f8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:46.831008 5338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b90b066ebf3433fb0df9dd90865bc62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.835179 5335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 603ab05a512d4ed1bcac2a5b86d6ee7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.838949 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54cd7746bb6a4252b74517999782bbab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.855681 8403 raft_consensus.cc:482] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:46.856313 8403 raft_consensus.cc:504] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.863705 8306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 438) took 1202 ms. Trace:
I20260808 13:35:46.784840 6046 ts_tablet_manager.cc:1428] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.181s user 0.005s sys 0.001s
I20260808 13:35:46.785259 5351 tablet_service.cc:1467] Processing CreateTablet for tablet 932a328069bd4edf8137f49a5da66a22 (DEFAULT_TABLE table=default.Table_0fdee57de12549eab898e92f30fe691d [id=9ad354d775a84d32bd5eb627356e0a6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.782171 5356 tablet_service.cc:1467] Processing CreateTablet for tablet 0db761bf06ae47fe8ab1eea6de408e6f (DEFAULT_TABLE table=default.Table_efea7e3ddc70498f94c05ddee5ba3d76 [id=c965816b949348ba98967bdc0115eefa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.881381 6046 tablet_bootstrap.cc:492] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:46.881524 5356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db761bf06ae47fe8ab1eea6de408e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.887444 5351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 932a328069bd4edf8137f49a5da66a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.888617 8403 leader_election.cc:288] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:46.861328 8407 tablet.cc:1722] T f85b4fd30d6c4903946891a399acf7a0 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260808 13:35:46.901131 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 203835us
I20260808 13:35:46.904641 5285 leader_election.cc:302] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:46.905622 8237 consensus_queue.cc:1035] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:46.912145 8230 raft_consensus.cc:2781] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:46.912750 8230 raft_consensus.cc:686] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:46.914038 8230 consensus_queue.cc:237] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:46.926855 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 603ab05a512d4ed1bcac2a5b86d6ee7d (DEFAULT_TABLE table=default.Table_0e051b613c6e452f89c46cb5aff7f7dd [id=5b28fc73db9a4600bfa8c43e9929b4cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:46.928088 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 603ab05a512d4ed1bcac2a5b86d6ee7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:35:46.943567 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:46.946254 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362dbd6990e740b39690537af98aa2f8" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:46.981489 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:46.987267 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249dcaf769934b75945d79d827a96be6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:46.987594 7741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54548 (request call id 0) took 7839 ms. Trace:
I20260808 13:35:46.988444 5866 raft_consensus.cc:3037] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:46.998497 8409 tablet.cc:1722] T adf9a03742a14cfe867c578c466838de P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:47.007551 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d31516006bc4a65b496dcf363db464a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:47.011099 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:47.012528 5835 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef8c23840c848ddb7d8b1023a839e94 (DEFAULT_TABLE table=default.Table_52f5a796f61f4be79a196bc23065491b [id=ee175d53df794e6aaf29faff1fc2bb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:35:47.013682 5835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef8c23840c848ddb7d8b1023a839e94. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:35:47.016654 5550 leader_election.cc:341] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.017154 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d31516006bc4a65b496dcf363db464a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:47.017596 5550 leader_election.cc:341] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.038415 6033 tablet_bootstrap.cc:654] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:47.010401 8416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:47.053253 8416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:47.062772 6026 tablet_bootstrap.cc:654] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:47.016021 5826 consensus_meta.cc:287] T 0b96fcd7da1c4e52bcad3779a23e141c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.601s user 0.013s sys 0.004s
W20260808 13:35:47.080127 5772 consensus_peers.cc:487] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:47.086650 5283 leader_election.cc:302] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:47.089815 5283 leader_election.cc:302] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:46.999509 4835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_cfa1c9cda5c140d2920999367a8c9509" } modify_external_catalogs: true
I20260808 13:35:47.099634 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:47.088364 8230 raft_consensus.cc:2726] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 13:35:46.864414 8306 rpcz_store.cc:276] 0808 13:35:45.661405 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.664240 (+ 2835us) service_pool.cc:227] Handling call
0808 13:35:46.312250 (+648010us) ts_tablet_manager.cc:1145] Deleting tablet 362dbd6990e740b39690537af98aa2f8
0808 13:35:46.312301 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:46.505156 (+192855us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:46.787929 (+282773us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:46.863610 (+ 75681us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":647199}
I20260808 13:35:47.107892 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 362dbd6990e740b39690537af98aa2f8 (table default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]) successfully deleted
I20260808 13:35:47.110116 8356 raft_consensus.cc:482] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:47.111009 8356 raft_consensus.cc:504] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.115622 8356 leader_election.cc:288] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:47.119602 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 489 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:47.120096 4832 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 496) took 2777 ms. Trace:
I20260808 13:35:47.104578 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:47.122995 4839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 16736500f94c43d3a3163f61b7606aad
W20260808 13:35:47.127374 8370 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.503s user 0.009s sys 0.004s
I20260808 13:35:47.123369 4835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_cfa1c9cda5c140d2920999367a8c9509" } modify_external_catalogs: true
I20260808 13:35:47.120446 4832 rpcz_store.cc:276] 0808 13:35:44.343126 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:46.116687 (+1773561us) service_pool.cc:227] Handling call
0808 13:35:46.120194 (+ 3507us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:46.488811 (+368617us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:47.077670 (+588859us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:47.119692 (+ 42022us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:47.119994 (+ 302us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:46.855635 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:46.861462 (+ 5827us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:46.861528 (+ 66us) write_op.cc:435] Acquired schema lock
0808 13:35:46.861542 (+ 14us) tablet.cc:660] Decoding operations
0808 13:35:46.864419 (+ 2877us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:46.864519 (+ 100us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:46.864576 (+ 57us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:46.865066 (+ 490us) tablet.cc:699] Row locks acquired
0808 13:35:46.865121 (+ 55us) write_op.cc:260] PREPARE: finished
0808 13:35:46.865349 (+ 228us) write_op.cc:270] Start()
0808 13:35:46.865550 (+ 201us) write_op.cc:276] Timestamp: P: 1786196146865235 usec, L: 0
0808 13:35:46.865578 (+ 28us) op_driver.cc:331] REPLICATION: starting
0808 13:35:47.001851 (+136273us) log.cc:842] Serialized 433 byte log entry
0808 13:35:47.006319 (+ 4468us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:47.049775 (+ 43456us) op_driver.cc:444] REPLICATION: finished
0808 13:35:47.064635 (+ 14860us) write_op.cc:301] APPLY: starting
0808 13:35:47.074456 (+ 9821us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:47.074863 (+ 407us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:47.074924 (+ 61us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:47.076041 (+ 1117us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:47.076406 (+ 365us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:47.076524 (+ 118us) write_op.cc:312] APPLY: finished
0808 13:35:47.076941 (+ 417us) log.cc:842] Serialized 29 byte log entry
0808 13:35:47.077216 (+ 275us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.077327 (+ 111us) write_op.cc:454] Released schema lock
0808 13:35:47.077378 (+ 51us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:47.078443 (+ 1065us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.078589 (+ 146us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":97,"apply.run_cpu_time_us":3795,"apply.run_wall_time_us":14351,"mutex_wait_us":24307,"num_ops":2,"prepare.queue_time_us":365161,"raft.queue_time_us":58873,"replication_time_us":171955,"spinlock_wait_cycles":348759680,"threads_started":1}]]}
I20260808 13:35:47.129805 5592 tablet_service.cc:1514] Processing DeleteTablet for tablet 16736500f94c43d3a3163f61b7606aad with delete_type TABLET_DATA_DELETED (replaced by tablet 16a055fbb45844a89991d6972619eef4) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:47.130688 4839 catalog_manager.cc:6565] ReplaceTablet: tablet 16736500f94c43d3a3163f61b7606aad deleted and replaced by tablet 16a055fbb45844a89991d6972619eef4
I20260808 13:35:47.131032 4839 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60270 (request call id 494) took 3340 ms. Trace:
I20260808 13:35:47.137316 8306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 435) took 1476 ms. Trace:
I20260808 13:35:47.138866 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 16736500f94c43d3a3163f61b7606aad with delete_type TABLET_DATA_DELETED (replaced by tablet 16a055fbb45844a89991d6972619eef4) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:47.137789 8306 rpcz_store.cc:276] 0808 13:35:45.660400 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.663574 (+ 3174us) service_pool.cc:227] Handling call
0808 13:35:47.136828 (+1473254us) ts_tablet_manager.cc:1145] Deleting tablet 196d80d806b0432cacf2ca6aa0da8c6b
0808 13:35:47.136950 (+ 122us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:47.137248 (+ 298us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1461441}
W20260808 13:35:47.139806 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 196d80d806b0432cacf2ca6aa0da8c6b because the tablet was not found. No further retry: Not found: Tablet not found: 196d80d806b0432cacf2ca6aa0da8c6b
I20260808 13:35:47.145349 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 489 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:47.146938 5829 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 452) took 1302 ms. Trace:
I20260808 13:35:47.147508 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:47.147420 5829 rpcz_store.cc:276] 0808 13:35:45.844491 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.884923 (+ 40432us) service_pool.cc:227] Handling call
0808 13:35:45.908262 (+ 23339us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:45.908350 (+ 88us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:46.479671 (+571321us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:46.614865 (+135194us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:47.146534 (+531669us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b5800001898. stack: 00007f19df247033 00007f19df246f30 00007f19dedc8534 00000000004db2b4 00000000004db249 00007f19df0bdf11 00007f19df241851 00007f19df241e82 00007f19e81018b9 00007f19e71355d8 00007f19e71f9063 00007f19e71f4e2f 00007f19e71f55ed 00007f19e71a6869 00007f19e5898c44 00007f19e5898bd2
0808 13:35:47.146877 (+ 343us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":646182272}
W20260808 13:35:47.151310 5841 consensus_meta.cc:287] T 54cd7746bb6a4252b74517999782bbab P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.576s user 0.005s sys 0.012s
I20260808 13:35:47.159934 8416 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:47.165648 8303 ts_tablet_manager.cc:1918] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:47.166302 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:47.172371 4841 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 493) took 3493 ms. Trace:
I20260808 13:35:47.176811 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:47.174108 4841 rpcz_store.cc:276] 0808 13:35:43.679399 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.969822 (+2290423us) service_pool.cc:227] Handling call
0808 13:35:45.976673 (+ 6851us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b4000001378. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227511ef11 00007f22752a2851 00007f22752a2702 00007f227e139b09 00007f227e139aa0 00007f227e275534 00007f227e1238b1 00007f227e0dcb31 00007f227e1ff18f 00007f227bd3c034 00007f227bd3bfc2
0808 13:35:46.060525 (+ 83852us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:46.080648 (+ 20123us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:47.171419 (+1090771us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:47.171970 (+ 551us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:47.172299 (+ 329us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:46.525158 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:46.526254 (+ 1096us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:46.526314 (+ 60us) write_op.cc:435] Acquired schema lock
0808 13:35:46.526363 (+ 49us) tablet.cc:660] Decoding operations
0808 13:35:46.527294 (+ 931us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:46.527345 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:46.527379 (+ 34us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:46.527811 (+ 432us) tablet.cc:699] Row locks acquired
0808 13:35:46.527828 (+ 17us) write_op.cc:260] PREPARE: finished
0808 13:35:46.528058 (+ 230us) write_op.cc:270] Start()
0808 13:35:46.528229 (+ 171us) write_op.cc:276] Timestamp: P: 1786196146527989 usec, L: 0
0808 13:35:46.528281 (+ 52us) op_driver.cc:331] REPLICATION: starting
0808 13:35:46.529297 (+ 1016us) log.cc:842] Serialized 433 byte log entry
0808 13:35:46.934146 (+404849us) op_driver.cc:444] REPLICATION: finished
0808 13:35:46.991869 (+ 57723us) write_op.cc:301] APPLY: starting
0808 13:35:46.992148 (+ 279us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:46.992478 (+ 330us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:47.035503 (+ 43025us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:47.058991 (+ 23488us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:47.082051 (+ 23060us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:47.082191 (+ 140us) write_op.cc:312] APPLY: finished
0808 13:35:47.086674 (+ 4483us) log.cc:842] Serialized 29 byte log entry
0808 13:35:47.087381 (+ 707us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.087739 (+ 358us) write_op.cc:454] Released schema lock
0808 13:35:47.087850 (+ 111us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":6670208,"child_traces":[["op",{"apply.queue_time_us":57329,"mutex_wait_us":386142,"num_ops":2,"prepare.queue_time_us":442881,"prepare.run_cpu_time_us":4574,"prepare.run_wall_time_us":320874,"raft.queue_time_us":26927,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":2566,"replication_time_us":405708,"thread_start_us":55810,"threads_started":1}]]}
I20260808 13:35:47.167647 5826 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 451) took 1347 ms. Trace:
I20260808 13:35:47.131407 4839 rpcz_store.cc:276] 0808 13:35:43.790418 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.978389 (+2187971us) service_pool.cc:227] Handling call
0808 13:35:47.130978 (+1152589us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:46.457647 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:46.458565 (+ 918us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:46.458631 (+ 66us) write_op.cc:435] Acquired schema lock
0808 13:35:46.458671 (+ 40us) tablet.cc:660] Decoding operations
0808 13:35:46.459440 (+ 769us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:46.459523 (+ 83us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:46.459569 (+ 46us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:46.460063 (+ 494us) tablet.cc:699] Row locks acquired
0808 13:35:46.460092 (+ 29us) write_op.cc:260] PREPARE: finished
0808 13:35:46.460336 (+ 244us) write_op.cc:270] Start()
0808 13:35:46.460489 (+ 153us) write_op.cc:276] Timestamp: P: 1786196146460199 usec, L: 0
0808 13:35:46.460545 (+ 56us) op_driver.cc:331] REPLICATION: starting
0808 13:35:46.523504 (+ 62959us) log.cc:842] Serialized 622 byte log entry
0808 13:35:46.834798 (+311294us) op_driver.cc:444] REPLICATION: finished
0808 13:35:46.924131 (+ 89333us) write_op.cc:301] APPLY: starting
0808 13:35:46.924242 (+ 111us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:46.924472 (+ 230us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:46.924859 (+ 387us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:46.925765 (+ 906us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:46.926229 (+ 464us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:46.926329 (+ 100us) write_op.cc:312] APPLY: finished
0808 13:35:46.926693 (+ 364us) log.cc:842] Serialized 29 byte log entry
0808 13:35:46.926970 (+ 277us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:46.927054 (+ 84us) write_op.cc:454] Released schema lock
0808 13:35:46.927137 (+ 83us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:46.990656 (+ 63519us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:46.990929 (+ 273us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":77822,"apply.run_cpu_time_us":3869,"apply.run_wall_time_us":82878,"mutex_wait_us":398178,"num_ops":2,"prepare.queue_time_us":398416,"prepare.run_cpu_time_us":4435,"prepare.run_wall_time_us":67344,"raft.queue_time_us":103,"raft.run_cpu_time_us":1209,"raft.run_wall_time_us":1209,"replication_time_us":354526,"spinlock_wait_cycles":113024,"thread_start_us":68879,"threads_started":1}]]}
I20260808 13:35:47.179740 4841 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60270:
table { table_name: "default.Table_a4c8baa577ff42f4b8f9aafb6ec9e6ec" } new_table_name: "default.Table_f1ad878382f04d038b6b08429d6b9db9" modify_external_catalogs: true
I20260808 13:35:47.182581 8306 tablet_replica.cc:330] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:47.183583 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_5d7feb173847448393c48160e861f0c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:47.193998 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.164s user 0.003s sys 0.000s
I20260808 13:35:47.194545 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0b76c4fdd94a6e9d9bb7709ff1b4be" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:47.196996 5385 raft_consensus.cc:1226] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Rejecting Update request from peer 98dd96f425fc4ab3a6a54f2dfaaf9fab for earlier term 1. Current term is 2. Ops: []
I20260808 13:35:47.197670 6046 tablet_bootstrap.cc:654] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:47.203269 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d7feb173847448393c48160e861f0c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:47.206700 8306 raft_consensus.cc:2229] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:47.177860 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
I20260808 13:35:47.217219 4840 catalog_manager.cc:5495] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } }
I20260808 13:35:47.219594 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 52deb2e968c04b99a436e04ae78d9969 (table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445]). Expected version 2 got 0
I20260808 13:35:47.220616 4840 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f85b4fd30d6c4903946891a399acf7a0 (table default.Table_f042821be4a8421ea3f64f33a071b22d [id=ea2ba343c17d403d8cb0bf9d80bee453]). Expected version 2 got 0
I20260808 13:35:47.207800 8306 raft_consensus.cc:2258] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:47.232801 8409 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:54830 (request call id 454) took 1356 ms. Trace:
I20260808 13:35:47.231657 8323 raft_consensus.cc:482] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:47.234196 8323 raft_consensus.cc:504] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.188002 8424 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:47.171511 8303 log.cc:1196] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/369a942248bd4cec9ee9170fe55601a3
W20260808 13:35:47.170567 5831 consensus_meta.cc:287] T b3d489bfe6724b74a830217bfc45a84a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.694s user 0.007s sys 0.015s
I20260808 13:35:47.272573 8434 tablet.cc:1722] T adf9a03742a14cfe867c578c466838de P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:47.277902 8393 raft_consensus.cc:482] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:47.278770 8356 raft_consensus.cc:482] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:47.279511 8356 raft_consensus.cc:504] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:47.282951 8393 raft_consensus.cc:504] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.284029 5866 raft_consensus.cc:2454] T 249dcaf769934b75945d79d827a96be6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:47.284642 8418 tablet.cc:1722] T f85b4fd30d6c4903946891a399acf7a0 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:47.287930 8356 leader_election.cc:288] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:47.289489 8393 leader_election.cc:288] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:47.288817 5842 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 460) took 1126 ms. Trace:
I20260808 13:35:47.290437 5842 rpcz_store.cc:276] 0808 13:35:46.162249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:46.169453 (+ 7204us) service_pool.cc:227] Handling call
0808 13:35:46.236708 (+ 67255us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:46.236810 (+ 102us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:47.083423 (+846613us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.231975 (+148552us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:47.288754 (+ 56779us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:47.290949 5882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09eb729339e47eb8db88eb6509daab4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:47.292268 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e4af746774747a70d425068e8dd63" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:47.293293 5877 raft_consensus.cc:2454] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 0.
W20260808 13:35:47.295146 5550 leader_election.cc:341] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.295740 5550 leader_election.cc:302] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:47.296617 8403 raft_consensus.cc:2781] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:47.297142 8403 raft_consensus.cc:482] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:47.252611 8106 consensus_queue.cc:1035] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.045s
I20260808 13:35:47.297729 8403 raft_consensus.cc:3037] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:47.300452 8323 leader_election.cc:288] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:47.303864 5827 consensus_meta.cc:287] T 0bf47d3df0ac480bb753d6dc46b4d7db P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent flushing consensus metadata: real 0.655s user 0.011s sys 0.008s
I20260808 13:35:47.303846 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e4af746774747a70d425068e8dd63" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:47.305289 5827 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 455) took 1375 ms. Trace:
I20260808 13:35:47.179641 5826 rpcz_store.cc:276] 0808 13:35:45.820194 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.820344 (+ 150us) service_pool.cc:227] Handling call
0808 13:35:45.823067 (+ 2723us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:45.823161 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:46.407198 (+584037us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.120971 (+713773us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:47.167560 (+ 46589us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:47.257858 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0b76c4fdd94a6e9d9bb7709ff1b4be" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:47.313858 8303 ts_tablet_manager.cc:1939] T 369a942248bd4cec9ee9170fe55601a3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:47.314991 8303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 444) took 1595 ms. Trace:
I20260808 13:35:47.315496 8303 rpcz_store.cc:276] 0808 13:35:45.719047 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.719144 (+ 97us) service_pool.cc:227] Handling call
0808 13:35:45.720840 (+ 1696us) ts_tablet_manager.cc:1145] Deleting tablet 369a942248bd4cec9ee9170fe55601a3
0808 13:35:45.720876 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:46.220444 (+499568us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.165238 (+944794us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.314905 (+149667us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":40344,"tablet-delete.queue_time_us":108}
I20260808 13:35:47.318135 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 369a942248bd4cec9ee9170fe55601a3 (table default.Table_7d31aaee9dd04cabbbbca675bde3a2f4 [id=e9b13612cdc742eb88f83d7ce4e999fd]) successfully deleted
I20260808 13:35:47.319224 5831 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54830 (request call id 457) took 1290 ms. Trace:
I20260808 13:35:47.323900 8403 raft_consensus.cc:504] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.325788 5640 raft_consensus.cc:2379] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in this term.
I20260808 13:35:47.330385 6026 tablet_bootstrap.cc:492] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:47.330801 6026 ts_tablet_manager.cc:1397] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.524s user 0.006s sys 0.015s
I20260808 13:35:47.319761 5831 rpcz_store.cc:276] 0808 13:35:46.028402 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:46.028579 (+ 177us) service_pool.cc:227] Handling call
0808 13:35:46.031844 (+ 3265us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:46.031975 (+ 131us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:46.475731 (+443756us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.281256 (+805525us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:47.319169 (+ 37913us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:47.328997 4776 master-stress-test.cc:578] Killing test thread 1/10
I20260808 13:35:47.339221 8407 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:50600 (request call id 440) took 1542 ms. Trace:
W20260808 13:35:47.342880 5550 leader_election.cc:341] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.343135 8306 ts_tablet_manager.cc:1905] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:47.305908 5827 rpcz_store.cc:276] 0808 13:35:45.929633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.930073 (+ 440us) service_pool.cc:227] Handling call
0808 13:35:45.932543 (+ 2470us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0808 13:35:45.932671 (+ 128us) ts_tablet_manager.cc:676] Creating new metadata...
0808 13:35:46.551824 (+619153us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.304529 (+752705us) tablet_replica.cc:205] Creating consensus instance
0808 13:35:47.305196 (+ 667us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:47.355409 8403 leader_election.cc:288] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:47.355613 8303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 445) took 1633 ms. Trace:
I20260808 13:35:47.339884 8407 rpcz_store.cc:276] 0808 13:35:45.797077 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.797682 (+ 605us) service_pool.cc:227] Handling call
0808 13:35:47.339123 (+1541441us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:45.798627 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:45.799274 (+ 647us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:45.799337 (+ 63us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:45.799442 (+ 105us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:45.799546 (+ 104us) alter_schema_op.cc:126] START. Timestamp: 1786196145799475
0808 13:35:45.799592 (+ 46us) op_driver.cc:331] REPLICATION: starting
0808 13:35:45.800365 (+ 773us) log.cc:842] Serialized 173 byte log entry
0808 13:35:46.831616 (+1031251us) op_driver.cc:444] REPLICATION: finished
0808 13:35:46.861128 (+ 29512us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:46.891091 (+ 29963us) log.cc:842] Serialized 5 byte log entry
0808 13:35:47.265889 (+374798us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.337029 (+ 71140us) log.cc:842] Serialized 14 byte log entry
0808 13:35:47.338827 (+ 1798us) alter_schema_op.cc:71] Released schema lock
0808 13:35:47.338880 (+ 53us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":29137,"mutex_wait_us":114,"prepare.queue_time_us":87,"prepare.run_cpu_time_us":8564,"prepare.run_wall_time_us":21145,"raft.queue_time_us":7414,"raft.run_cpu_time_us":1656,"raft.run_wall_time_us":18601,"replication_time_us":1031890,"thread_start_us":210513,"threads_started":3,"wal-append.queue_time_us":178219,"wal-append.run_cpu_time_us":2290,"wal-append.run_wall_time_us":1013139}]]}
I20260808 13:35:47.334522 6026 raft_consensus.cc:348] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:47.359081 6026 raft_consensus.cc:374] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:47.359328 6026 raft_consensus.cc:729] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
W20260808 13:35:47.367903 5551 leader_election.cc:341] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.368273 5551 leader_election.cc:302] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:47.370751 8356 raft_consensus.cc:2726] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:47.374670 6026 consensus_queue.cc:260] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:47.375120 8393 raft_consensus.cc:482] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:47.376490 6026 ts_tablet_manager.cc:1428] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.045s user 0.007s sys 0.000s
I20260808 13:35:47.375809 8393 raft_consensus.cc:504] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.376482 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e4af746774747a70d425068e8dd63" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:47.379667 8393 leader_election.cc:288] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:47.310075 8419 tablet.cc:1722] T f85b4fd30d6c4903946891a399acf7a0 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:47.399464 8280 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: acquiring CA information for follower catalog manager: success
I20260808 13:35:47.400389 6026 tablet_bootstrap.cc:492] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:47.406241 8363 ts_tablet_manager.cc:1918] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:47.406774 8363 log.cc:1196] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/369a942248bd4cec9ee9170fe55601a3
I20260808 13:35:47.411877 8363 ts_tablet_manager.cc:1939] T 369a942248bd4cec9ee9170fe55601a3 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:47.424393 8371 raft_consensus.cc:482] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:47.424721 8363 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 458) took 1555 ms. Trace:
I20260808 13:35:47.425143 8371 raft_consensus.cc:504] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.425397 8363 rpcz_store.cc:276] 0808 13:35:45.869765 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.870098 (+ 333us) service_pool.cc:227] Handling call
0808 13:35:46.379546 (+509448us) ts_tablet_manager.cc:1145] Deleting tablet 369a942248bd4cec9ee9170fe55601a3
0808 13:35:46.390269 (+ 10723us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:46.962964 (+572695us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.405793 (+442829us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.424593 (+ 18800us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":492064}
I20260808 13:35:47.425925 8312 consensus_queue.cc:1046] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:47.429425 8323 raft_consensus.cc:3032] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Stepping down as leader of term 1
I20260808 13:35:47.430022 8323 raft_consensus.cc:729] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:47.431608 8323 consensus_queue.cc:260] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.433779 8323 raft_consensus.cc:3037] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:47.249636 8409 rpcz_store.cc:276] 0808 13:35:45.876129 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.876438 (+ 309us) service_pool.cc:227] Handling call
0808 13:35:47.232699 (+1356261us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:45.878213 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:45.878845 (+ 632us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:45.878880 (+ 35us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:45.879003 (+ 123us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:45.879188 (+ 185us) alter_schema_op.cc:126] START. Timestamp: 1786196145879048
0808 13:35:45.879205 (+ 17us) op_driver.cc:331] REPLICATION: starting
0808 13:35:45.880085 (+ 880us) log.cc:842] Serialized 173 byte log entry
0808 13:35:46.852928 (+972843us) op_driver.cc:444] REPLICATION: finished
0808 13:35:46.997412 (+144484us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:47.003491 (+ 6079us) log.cc:842] Serialized 5 byte log entry
0808 13:35:47.177924 (+174433us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.179974 (+ 2050us) log.cc:842] Serialized 14 byte log entry
0808 13:35:47.232397 (+ 52423us) alter_schema_op.cc:71] Released schema lock
0808 13:35:47.232474 (+ 77us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":143943,"mutex_wait_us":142250,"prepare.queue_time_us":217,"prepare.run_cpu_time_us":18024,"prepare.run_wall_time_us":360738,"raft.queue_time_us":150564,"raft.run_cpu_time_us":1155,"raft.run_wall_time_us":99971,"replication_time_us":973595,"thread_start_us":309226,"threads_started":2,"wal-append.queue_time_us":211790}]]}
I20260808 13:35:47.356335 8303 rpcz_store.cc:276] 0808 13:35:45.722569 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.723011 (+ 442us) service_pool.cc:227] Handling call
0808 13:35:47.355160 (+1632149us) ts_tablet_manager.cc:1145] Deleting tablet 8b57966a9b7b4673ab9f98720cfd456d
0808 13:35:47.355235 (+ 75us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:47.355522 (+ 287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1601001}
I20260808 13:35:47.439013 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e4af746774747a70d425068e8dd63" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:47.439762 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 8b57966a9b7b4673ab9f98720cfd456d because the tablet was not found. No further retry: Not found: Tablet not found: 8b57966a9b7b4673ab9f98720cfd456d
W20260808 13:35:47.440342 5771 consensus_peers.cc:487] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:47.441120 5771 leader_election.cc:341] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.441347 8303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 448) took 1642 ms. Trace:
I20260808 13:35:47.444077 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet 16736500f94c43d3a3163f61b7606aad with delete_type TABLET_DATA_DELETED (replaced by tablet 16a055fbb45844a89991d6972619eef4) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:47.444118 8303 rpcz_store.cc:276] 0808 13:35:45.799425 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.799623 (+ 198us) service_pool.cc:227] Handling call
0808 13:35:47.440974 (+1641351us) ts_tablet_manager.cc:1145] Deleting tablet f0ba43d1479a4f9e9d08bdfa29e7157b
0808 13:35:47.441045 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:47.441283 (+ 238us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1640245}
I20260808 13:35:47.445906 8403 raft_consensus.cc:482] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:47.446532 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 369a942248bd4cec9ee9170fe55601a3 (table default.Table_7d31aaee9dd04cabbbbca675bde3a2f4 [id=e9b13612cdc742eb88f83d7ce4e999fd]) successfully deleted
W20260808 13:35:47.446736 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet f0ba43d1479a4f9e9d08bdfa29e7157b because the tablet was not found. No further retry: Not found: Tablet not found: f0ba43d1479a4f9e9d08bdfa29e7157b
I20260808 13:35:47.441489 5771 leader_election.cc:302] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:47.446710 8403 raft_consensus.cc:504] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.450330 8379 raft_consensus.cc:3037] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:47.407399 8237 raft_consensus.cc:2726] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:47.458206 6026 tablet_bootstrap.cc:654] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:47.455474 8303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 449) took 1645 ms. Trace:
I20260808 13:35:47.459271 8303 rpcz_store.cc:276] 0808 13:35:45.810002 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.810126 (+ 124us) service_pool.cc:227] Handling call
0808 13:35:47.454988 (+1644862us) ts_tablet_manager.cc:1145] Deleting tablet d8b07b7d31ef41e484389c0f007598f0
0808 13:35:47.455064 (+ 76us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:47.455383 (+ 319us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1643940}
W20260808 13:35:47.461508 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet d8b07b7d31ef41e484389c0f007598f0 because the tablet was not found. No further retry: Not found: Tablet not found: d8b07b7d31ef41e484389c0f007598f0
I20260808 13:35:47.462818 8303 tablet_replica.cc:330] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:47.464792 8303 raft_consensus.cc:2229] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:47.470492 8403 leader_election.cc:288] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:47.482836 6033 tablet_bootstrap.cc:492] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:47.483359 6033 ts_tablet_manager.cc:1397] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.742s user 0.011s sys 0.022s
I20260808 13:35:47.488869 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e4af746774747a70d425068e8dd63" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:47.489210 6033 raft_consensus.cc:348] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.490115 6033 raft_consensus.cc:374] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:47.490538 6033 raft_consensus.cc:729] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:47.493587 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a73e98a8b20144508a5a8d451f4bb302" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:47.495003 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64888f0d27c94ee7b25c0dda57e766f6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:47.491370 6033 consensus_queue.cc:260] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.496641 8371 leader_election.cc:288] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:47.500731 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.174s user 0.002s sys 0.001s
I20260808 13:35:47.507792 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a73e98a8b20144508a5a8d451f4bb302" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:47.509869 5550 leader_election.cc:341] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.510537 5630 raft_consensus.cc:2454] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:47.521695 8363 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 460) took 1649 ms. Trace:
W20260808 13:35:47.510560 5550 leader_election.cc:341] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.514564 8306 ts_tablet_manager.cc:1918] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:47.524777 8306 log.cc:1196] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/daa59a18ae624bc8975bee2c460895a7
I20260808 13:35:47.526135 8306 ts_tablet_manager.cc:1939] T daa59a18ae624bc8975bee2c460895a7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:47.530372 6033 ts_tablet_manager.cc:1428] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.047s user 0.008s sys 0.001s
I20260808 13:35:47.531538 6033 tablet_bootstrap.cc:492] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:47.532445 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a73e98a8b20144508a5a8d451f4bb302" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:47.545244 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 175613us
I20260808 13:35:47.523056 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09eb729339e47eb8db88eb6509daab4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:47.527868 5285 leader_election.cc:302] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
W20260808 13:35:47.575525 5283 leader_election.cc:341] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:47.577960 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 50174.2us
I20260808 13:35:47.588032 8379 raft_consensus.cc:2726] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:47.588246 8338 raft_consensus.cc:2781] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:47.589156 8338 raft_consensus.cc:482] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:47.589284 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64888f0d27c94ee7b25c0dda57e766f6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:47.589758 8303 raft_consensus.cc:2258] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:47.589836 8338 raft_consensus.cc:3037] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:47.579335 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e4af746774747a70d425068e8dd63" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:47.601007 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a73e98a8b20144508a5a8d451f4bb302" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:47.609582 6026 tablet_bootstrap.cc:492] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:47.610014 6026 ts_tablet_manager.cc:1397] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.211s user 0.014s sys 0.007s
I20260808 13:35:47.612870 8306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 436) took 1952 ms. Trace:
W20260808 13:35:47.615407 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.071s user 0.001s sys 0.000s
I20260808 13:35:47.574255 8363 rpcz_store.cc:276] 0808 13:35:45.871853 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.872095 (+ 242us) service_pool.cc:227] Handling call
0808 13:35:47.475821 (+1603726us) ts_tablet_manager.cc:1145] Deleting tablet d8b07b7d31ef41e484389c0f007598f0
0808 13:35:47.481360 (+ 5539us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:47.521610 (+ 40250us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1539934}
I20260808 13:35:47.621714 4835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 26ce3623d547476684782fd395fb8fcf
I20260808 13:35:47.623476 4835 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 498) took 2295 ms. Trace:
I20260808 13:35:47.624583 6026 raft_consensus.cc:348] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.625352 6026 raft_consensus.cc:374] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:47.625600 6026 raft_consensus.cc:729] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:47.628206 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ce3623d547476684782fd395fb8fcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:47 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:47.613379 8306 rpcz_store.cc:276] 0808 13:35:45.660693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.665213 (+ 4520us) service_pool.cc:227] Handling call
0808 13:35:47.182263 (+1517050us) ts_tablet_manager.cc:1145] Deleting tablet daa59a18ae624bc8975bee2c460895a7
0808 13:35:47.182353 (+ 90us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:47.418259 (+235906us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.514137 (+ 95878us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.612762 (+ 98625us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1516498}
I20260808 13:35:47.631323 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ce3623d547476684782fd395fb8fcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:47 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:47.631675 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet d8b07b7d31ef41e484389c0f007598f0 because the tablet was not found. No further retry: Not found: Tablet not found: d8b07b7d31ef41e484389c0f007598f0
I20260808 13:35:47.631930 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet daa59a18ae624bc8975bee2c460895a7 (table default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]) successfully deleted
I20260808 13:35:47.631876 8306 tablet_replica.cc:330] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:47.633875 8306 raft_consensus.cc:2229] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:47.634704 8306 raft_consensus.cc:2258] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:47.623799 4835 rpcz_store.cc:276] 0808 13:35:45.327846 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:46.785042 (+1457196us) service_pool.cc:227] Handling call
0808 13:35:46.999393 (+214351us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1ff466 00007f227bd40b34 00007f227bd40ac2 00007f227bd409f1 00007f227bd4096c 00007f227bd3fc22 00007f2279fde94c 00007f2279fddd86
0808 13:35:47.128182 (+128789us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:47.161275 (+ 33093us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:47.161460 (+ 185us) catalog_manager.cc:2761] Locking tablets
0808 13:35:47.162444 (+ 984us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:47.621206 (+458762us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:47.621483 (+ 277us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:47.621604 (+ 121us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:47.623394 (+ 1790us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:47.498032 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:47.499759 (+ 1727us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:47.499832 (+ 73us) write_op.cc:435] Acquired schema lock
0808 13:35:47.499956 (+ 124us) tablet.cc:660] Decoding operations
0808 13:35:47.527357 (+ 27401us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:47.527455 (+ 98us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:47.527495 (+ 40us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:47.528241 (+ 746us) tablet.cc:699] Row locks acquired
0808 13:35:47.528271 (+ 30us) write_op.cc:260] PREPARE: finished
0808 13:35:47.528541 (+ 270us) write_op.cc:270] Start()
0808 13:35:47.528730 (+ 189us) write_op.cc:276] Timestamp: P: 1786196147528441 usec, L: 0
0808 13:35:47.528783 (+ 53us) op_driver.cc:331] REPLICATION: starting
0808 13:35:47.534078 (+ 5295us) log.cc:842] Serialized 725 byte log entry
0808 13:35:47.618148 (+ 84070us) op_driver.cc:444] REPLICATION: finished
0808 13:35:47.618641 (+ 493us) write_op.cc:301] APPLY: starting
0808 13:35:47.618754 (+ 113us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:47.618892 (+ 138us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:47.619027 (+ 135us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:47.619445 (+ 418us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:47.619656 (+ 211us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:47.619754 (+ 98us) write_op.cc:312] APPLY: finished
0808 13:35:47.619974 (+ 220us) log.cc:842] Serialized 29 byte log entry
0808 13:35:47.620138 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.620253 (+ 115us) write_op.cc:454] Released schema lock
0808 13:35:47.620313 (+ 60us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:47.620650 (+ 337us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.620697 (+ 47us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":412334336,"child_traces":[["op",{"apply.queue_time_us":232,"apply.run_cpu_time_us":2338,"apply.run_wall_time_us":2337,"num_ops":2,"prepare.queue_time_us":330504,"prepare.run_cpu_time_us":5319,"prepare.run_wall_time_us":108283,"raft.queue_time_us":128,"raft.run_cpu_time_us":1248,"raft.run_wall_time_us":1247,"replication_time_us":89223,"spinlock_wait_cycles":48512}]]}
I20260808 13:35:47.641254 6033 tablet_bootstrap.cc:654] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:47.666007 5551 leader_election.cc:341] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.679754 5551 leader_election.cc:302] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:47.680635 8356 raft_consensus.cc:2726] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:47.690831 8409 tablet.cc:1722] T 234591e698f8472e90e1f610da37ff3a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:47.712746 6026 consensus_queue.cc:260] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:47.726812 5551 leader_election.cc:341] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.727362 5551 leader_election.cc:302] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:47.728248 8356 raft_consensus.cc:2726] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:47.728896 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ce3623d547476684782fd395fb8fcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:47 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:47.731276 6026 ts_tablet_manager.cc:1428] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.121s user 0.000s sys 0.010s
I20260808 13:35:47.732896 6026 tablet_bootstrap.cc:492] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:47.734117 4841 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 228ad48e495b45a29e96d427b7098497 consensus state has no leader
I20260808 13:35:47.734906 4841 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 499) took 1691 ms. Trace:
I20260808 13:35:47.735253 4841 rpcz_store.cc:276] 0808 13:35:46.043878 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.179220 (+1135342us) service_pool.cc:227] Handling call
0808 13:35:47.181910 (+ 2690us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:47.183639 (+ 1729us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:47.189327 (+ 5688us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:47.189635 (+ 308us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:47.683266 (+493631us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:47.734847 (+ 51581us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:47.614871 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:47.621817 (+ 6946us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:47.621851 (+ 34us) write_op.cc:435] Acquired schema lock
0808 13:35:47.621864 (+ 13us) tablet.cc:660] Decoding operations
0808 13:35:47.622251 (+ 387us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:47.622297 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:47.622320 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:47.622551 (+ 231us) tablet.cc:699] Row locks acquired
0808 13:35:47.622564 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:47.622862 (+ 298us) write_op.cc:270] Start()
0808 13:35:47.623029 (+ 167us) write_op.cc:276] Timestamp: P: 1786196147622764 usec, L: 0
0808 13:35:47.623045 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:47.663119 (+ 40074us) log.cc:842] Serialized 405 byte log entry
0808 13:35:47.664173 (+ 1054us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4471 00007f227c22b198 00007f227c68cade 00007f227c68c2b8 00007f227c690ba1 00007f227c690b59 00007f227c690ae9 00007f227c690ab1 00007f227c68fdad 0000000000580784 00000000005805b9
0808 13:35:47.679976 (+ 15803us) op_driver.cc:444] REPLICATION: finished
0808 13:35:47.680958 (+ 982us) write_op.cc:301] APPLY: starting
0808 13:35:47.681057 (+ 99us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:47.681208 (+ 151us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:47.681235 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:47.681510 (+ 275us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:47.681708 (+ 198us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:47.681758 (+ 50us) write_op.cc:312] APPLY: finished
0808 13:35:47.682049 (+ 291us) log.cc:842] Serialized 23 byte log entry
0808 13:35:47.682241 (+ 192us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.682322 (+ 81us) write_op.cc:454] Released schema lock
0808 13:35:47.682380 (+ 58us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:47.682720 (+ 340us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.682796 (+ 76us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":672,"apply.run_cpu_time_us":2097,"apply.run_wall_time_us":2095,"num_ops":1,"prepare.queue_time_us":415178,"prepare.run_cpu_time_us":4676,"prepare.run_wall_time_us":90389,"raft.queue_time_us":5369,"raft.run_cpu_time_us":1274,"raft.run_wall_time_us":1286,"replication_time_us":56766,"spinlock_wait_cycles":90229504}]]}
I20260808 13:35:47.742002 6046 tablet_bootstrap.cc:492] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:47.678442 8303 ts_tablet_manager.cc:1905] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:47.748687 8363 tablet_replica.cc:330] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:47.743587 6046 ts_tablet_manager.cc:1397] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.862s user 0.013s sys 0.020s
I20260808 13:35:47.755232 8363 raft_consensus.cc:2229] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:47.754863 8419 tablet.cc:1722] T adf9a03742a14cfe867c578c466838de P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260808 13:35:47.757220 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 228ad48e495b45a29e96d427b7098497 (table default.Table_f1ad878382f04d038b6b08429d6b9db9 [id=d84314027def49d380b8c13a0431f39e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 228ad48e495b45a29e96d427b7098497 consensus state has no leader
I20260808 13:35:47.757627 8363 raft_consensus.cc:2258] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:47.761194 8363 ts_tablet_manager.cc:1905] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:47.762806 5551 leader_election.cc:341] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.765417 4839 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60270 (request call id 500) took 1278 ms. Trace:
I20260808 13:35:47.765890 4839 rpcz_store.cc:276] 0808 13:35:46.487336 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.180613 (+693277us) service_pool.cc:227] Handling call
0808 13:35:47.212757 (+ 32144us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:35:47.361490 (+148733us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:35:47.764563 (+403073us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:35:47.764955 (+ 392us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:35:47.765354 (+ 399us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:47.705195 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:47.710062 (+ 4867us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:47.710333 (+ 271us) write_op.cc:435] Acquired schema lock
0808 13:35:47.710481 (+ 148us) tablet.cc:660] Decoding operations
0808 13:35:47.716919 (+ 6438us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:47.717111 (+ 192us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:47.717353 (+ 242us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:47.719175 (+ 1822us) tablet.cc:699] Row locks acquired
0808 13:35:47.719201 (+ 26us) write_op.cc:260] PREPARE: finished
0808 13:35:47.724577 (+ 5376us) write_op.cc:270] Start()
0808 13:35:47.725947 (+ 1370us) write_op.cc:276] Timestamp: P: 1786196147719814 usec, L: 0
0808 13:35:47.726284 (+ 337us) op_driver.cc:331] REPLICATION: starting
0808 13:35:47.742750 (+ 16466us) log.cc:842] Serialized 433 byte log entry
0808 13:35:47.761989 (+ 19239us) op_driver.cc:444] REPLICATION: finished
0808 13:35:47.762360 (+ 371us) write_op.cc:301] APPLY: starting
0808 13:35:47.762416 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:47.762541 (+ 125us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:47.762671 (+ 130us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:47.763203 (+ 532us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:47.763472 (+ 269us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:35:47.763521 (+ 49us) write_op.cc:312] APPLY: finished
0808 13:35:47.763765 (+ 244us) log.cc:842] Serialized 29 byte log entry
0808 13:35:47.763938 (+ 173us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.764010 (+ 72us) write_op.cc:454] Released schema lock
0808 13:35:47.764041 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:47.764222 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.764244 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":226,"apply.run_cpu_time_us":2030,"apply.run_wall_time_us":2030,"num_ops":2,"prepare.queue_time_us":208751,"raft.queue_time_us":7147,"raft.run_cpu_time_us":756,"raft.run_wall_time_us":756,"replication_time_us":35626}]]}
W20260808 13:35:47.774828 5551 leader_election.cc:341] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.775947 5551 leader_election.cc:302] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:47.792507 8356 raft_consensus.cc:2726] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:47.793417 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60270:
name: "default.Table_dbb6f93863704e4e9c573d989a367368"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:35:47.796587 5551 leader_election.cc:341] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:47.797410 5551 leader_election.cc:341] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.797750 5551 leader_election.cc:302] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:47.794361 8409 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:54830 (request call id 464) took 1259 ms. Trace:
I20260808 13:35:47.734402 5874 raft_consensus.cc:2454] T 2d8e4af746774747a70d425068e8dd63 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 1.
W20260808 13:35:47.811642 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 212814us
I20260808 13:35:47.790881 8452 tablet.cc:1722] T 234591e698f8472e90e1f610da37ff3a P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:47.815325 8356 raft_consensus.cc:2726] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:47.817946 8356 raft_consensus.cc:482] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:47.818746 8356 raft_consensus.cc:504] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.825398 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9e8d48c55349058cf80bf7c2e149e7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:47.750409 8306 ts_tablet_manager.cc:1905] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:47.826898 5550 leader_election.cc:341] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:47.832163 8356 leader_election.cc:288] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:47.835292 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.157s user 0.000s sys 0.004s
I20260808 13:35:47.796216 8338 raft_consensus.cc:504] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.846323 5383 raft_consensus.cc:2379] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:47.856266 4837 catalog_manager.cc:5495] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:47.857868 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 52deb2e968c04b99a436e04ae78d9969 (table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445]). Expected version 2 got 0
I20260808 13:35:47.866873 8306 ts_tablet_manager.cc:1918] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:47.867645 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:47.868094 8306 log.cc:1196] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/851a0b374a854efd87dd66e39b9df2d5
I20260808 13:35:47.868693 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:47.869475 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet f85b4fd30d6c4903946891a399acf7a0 (table default.Table_f042821be4a8421ea3f64f33a071b22d [id=ea2ba343c17d403d8cb0bf9d80bee453]). Expected version 2 got 0
I20260808 13:35:47.799819 6046 raft_consensus.cc:348] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.871031 6046 raft_consensus.cc:374] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:47.871351 6046 raft_consensus.cc:729] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:47.872653 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9e8d48c55349058cf80bf7c2e149e7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:47.883430 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbb6f93863704e4e9c573d989a367368 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:35:47.871068 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a73e98a8b20144508a5a8d451f4bb302" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:47.886511 5630 raft_consensus.cc:3037] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:47.891158 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 52133.3us
I20260808 13:35:47.887066 8306 ts_tablet_manager.cc:1939] T 851a0b374a854efd87dd66e39b9df2d5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:47.870198 4837 catalog_manager.cc:5495] T 3b1a27f687b54bc592ef4efb72b7247f P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:47.907080 6033 tablet_bootstrap.cc:492] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:47.907573 6033 ts_tablet_manager.cc:1397] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.376s user 0.003s sys 0.013s
I20260808 13:35:47.901119 8306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 437) took 2240 ms. Trace:
I20260808 13:35:47.908244 8306 rpcz_store.cc:276] 0808 13:35:45.661077 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.665800 (+ 4723us) service_pool.cc:227] Handling call
0808 13:35:47.631574 (+1965774us) ts_tablet_manager.cc:1145] Deleting tablet 851a0b374a854efd87dd66e39b9df2d5
0808 13:35:47.631672 (+ 98us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:47.853786 (+222114us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.866595 (+ 12809us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.901032 (+ 34437us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1965192}
I20260808 13:35:47.910286 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 851a0b374a854efd87dd66e39b9df2d5 (table default.Table_6ba381f21fc24362bc6d18499af1d329 [id=393e6ebfd240438297030391565be087]) successfully deleted
I20260808 13:35:47.807137 8409 rpcz_store.cc:276] 0808 13:35:46.535347 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:46.535453 (+ 106us) service_pool.cc:227] Handling call
0808 13:35:47.794295 (+1258842us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:46.550910 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:46.574492 (+ 23582us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:46.574540 (+ 48us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:46.577982 (+ 3442us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:46.578160 (+ 178us) alter_schema_op.cc:126] START. Timestamp: 1786196146556198
0808 13:35:46.578193 (+ 33us) op_driver.cc:331] REPLICATION: starting
0808 13:35:46.647381 (+ 69188us) log.cc:842] Serialized 173 byte log entry
0808 13:35:47.690053 (+1042672us) op_driver.cc:444] REPLICATION: finished
0808 13:35:47.690737 (+ 684us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:47.693102 (+ 2365us) log.cc:842] Serialized 5 byte log entry
0808 13:35:47.756212 (+ 63110us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:47.757901 (+ 1689us) log.cc:842] Serialized 14 byte log entry
0808 13:35:47.794138 (+ 36237us) alter_schema_op.cc:71] Released schema lock
0808 13:35:47.794165 (+ 27us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":317,"mutex_wait_us":27937,"prepare.queue_time_us":8413,"prepare.run_cpu_time_us":12606,"prepare.run_wall_time_us":463535,"raft.queue_time_us":28379,"raft.run_cpu_time_us":1741,"raft.run_wall_time_us":88722,"replication_time_us":1111680,"thread_start_us":303136,"threads_started":1,"wal-append.queue_time_us":347282}]]}
I20260808 13:35:47.918617 4776 master-stress-test.cc:580] Killed test thread 1/10
I20260808 13:35:47.918877 4776 master-stress-test.cc:578] Killing test thread 2/10
I20260808 13:35:47.919106 4776 master-stress-test.cc:580] Killed test thread 2/10
I20260808 13:35:47.831945 8418 tablet.cc:1722] T f66b38e83dab4bf1a3d2d1e088a96988 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:47.919286 4776 master-stress-test.cc:578] Killing test thread 3/10
I20260808 13:35:47.919494 4776 master-stress-test.cc:580] Killed test thread 3/10
I20260808 13:35:47.919672 4776 master-stress-test.cc:578] Killing test thread 4/10
I20260808 13:35:47.919900 4776 master-stress-test.cc:580] Killed test thread 4/10
I20260808 13:35:47.920101 4776 master-stress-test.cc:578] Killing test thread 5/10
W20260808 13:35:47.922322 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 74146.3us
I20260808 13:35:47.925343 5550 leader_election.cc:302] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:47.927353 8449 raft_consensus.cc:2781] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:47.927984 8449 raft_consensus.cc:686] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:47.929196 8449 consensus_queue.cc:237] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.931857 4837 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
I20260808 13:35:47.903594 5630 raft_consensus.cc:2454] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:47.948505 4839 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:34286 (request call id 2) took 2986 ms. Trace:
I20260808 13:35:47.948901 4839 rpcz_store.cc:276] 0808 13:35:44.962312 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.931670 (+2969358us) service_pool.cc:227] Handling call
0808 13:35:47.948071 (+ 16401us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e200c3a 00007f227bd6ae37 00007f227bd6adc2 00007f227bd6acf1 00007f227bd6ac6c 00007f227bd69f22 00007f2279fde94c 00007f2279fddd86
0808 13:35:47.948427 (+ 356us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":33889536}
I20260808 13:35:47.950704 6026 tablet_bootstrap.cc:654] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:47.938866 6046 consensus_queue.cc:260] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.957927 6046 ts_tablet_manager.cc:1428] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.205s user 0.002s sys 0.005s
I20260808 13:35:47.959468 8338 leader_election.cc:288] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:47.960705 5285 leader_election.cc:302] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
W20260808 13:35:47.962915 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.124s user 0.000s sys 0.001s
I20260808 13:35:47.969897 8450 raft_consensus.cc:482] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:47.970860 8338 raft_consensus.cc:2781] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:47.970494 8450 raft_consensus.cc:504] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.973541 8338 raft_consensus.cc:686] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:47.974933 8338 consensus_queue.cc:237] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:47.974300 8450 leader_election.cc:288] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:47.876049 5551 leader_election.cc:302] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:47.879104 8434 tablet.cc:1722] T 234591e698f8472e90e1f610da37ff3a P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:47.989393 8450 raft_consensus.cc:2726] T a73e98a8b20144508a5a8d451f4bb302 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:47.988390 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 112466us
I20260808 13:35:47.998798 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a73e98a8b20144508a5a8d451f4bb302" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:48.003759 5551 leader_election.cc:341] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.004179 5551 leader_election.cc:302] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:48.015057 6033 raft_consensus.cc:348] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.015801 6033 raft_consensus.cc:374] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.016044 6033 raft_consensus.cc:729] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:48.016631 6033 consensus_queue.cc:260] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.006929 6046 tablet_bootstrap.cc:492] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:48.026079 8306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 444) took 2154 ms. Trace:
I20260808 13:35:48.026515 8306 rpcz_store.cc:276] 0808 13:35:45.871394 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.871518 (+ 124us) service_pool.cc:227] Handling call
0808 13:35:48.025624 (+2154106us) ts_tablet_manager.cc:1145] Deleting tablet 553a01330cfe4dd38a61860dc943fbbc
0808 13:35:48.025676 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.026010 (+ 334us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2153386}
W20260808 13:35:48.027730 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 553a01330cfe4dd38a61860dc943fbbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 553a01330cfe4dd38a61860dc943fbbc already in progress: creating tablet
I20260808 13:35:48.028012 8306 tablet_replica.cc:330] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:48.028889 8306 raft_consensus.cc:2229] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:48.032446 6026 tablet_bootstrap.cc:492] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:48.048403 6026 ts_tablet_manager.cc:1397] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.316s user 0.019s sys 0.006s
W20260808 13:35:48.052752 5283 consensus_peers.cc:487] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:48.047274 8434 tablet.cc:1722] T 5663fa6a1bb148c3afc9340477b05a4b P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:48.055215 8237 consensus_queue.cc:1035] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:48.051318 8379 raft_consensus.cc:482] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.062737 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04d3dd028da4e7f86689e819548d0c5" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:48.080653 8379 raft_consensus.cc:504] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:48.097476 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.086s user 0.000s sys 0.002s
I20260808 13:35:48.105188 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8cf06462484c99a860762a82f7ad64" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:48.105870 5384 raft_consensus.cc:2454] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:48.107079 5771 leader_election.cc:302] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:48.107402 8280 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:35:48.108850 8370 raft_consensus.cc:2781] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:35:48.109392 8370 raft_consensus.cc:482] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:48.109802 8370 raft_consensus.cc:3037] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:48.139808 8476 tablet.cc:1722] T 5663fa6a1bb148c3afc9340477b05a4b P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260808 13:35:48.145785 5283 leader_election.cc:341] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.154896 8379 leader_election.cc:288] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:48.163311 6033 ts_tablet_manager.cc:1428] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.255s user 0.011s sys 0.002s
I20260808 13:35:48.164539 6033 tablet_bootstrap.cc:492] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:48.132494 8419 tablet.cc:1722] T f66b38e83dab4bf1a3d2d1e088a96988 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:48.192647 8450 raft_consensus.cc:2726] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:48.195099 8306 raft_consensus.cc:2258] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:48.197268 5550 leader_election.cc:341] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.197769 6046 tablet_bootstrap.cc:654] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:48.204699 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 200968us
I20260808 13:35:48.205878 8306 ts_tablet_manager.cc:1905] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:48.220690 8409 tablet.cc:1722] T 5663fa6a1bb148c3afc9340477b05a4b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:48.214013 8340 consensus_queue.cc:1035] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:48.233386 6026 raft_consensus.cc:348] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.234186 6026 raft_consensus.cc:374] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.234488 6026 raft_consensus.cc:729] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:48.235507 6026 consensus_queue.cc:260] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.251866 6026 ts_tablet_manager.cc:1428] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.201s user 0.008s sys 0.000s
I20260808 13:35:48.269171 8449 raft_consensus.cc:482] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.274952 8449 raft_consensus.cc:504] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.278475 8363 ts_tablet_manager.cc:1918] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:48.278982 8363 log.cc:1196] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/daa59a18ae624bc8975bee2c460895a7
I20260808 13:35:48.279382 8449 leader_election.cc:288] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:48.279810 8363 ts_tablet_manager.cc:1939] T daa59a18ae624bc8975bee2c460895a7 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:48.280561 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bed5e6d1aa49a9b14ba86403c985ce" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:48.280797 8363 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 462) took 2408 ms. Trace:
I20260808 13:35:48.281215 8363 rpcz_store.cc:276] 0808 13:35:45.872747 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.873433 (+ 686us) service_pool.cc:227] Handling call
0808 13:35:47.621300 (+1747867us) ts_tablet_manager.cc:1145] Deleting tablet daa59a18ae624bc8975bee2c460895a7
0808 13:35:47.748474 (+127174us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.065798 (+317324us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.278219 (+212421us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.280686 (+ 2467us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4839,"tablet-delete.queue_time_us":1733228}
W20260808 13:35:48.282157 5550 leader_election.cc:341] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.282954 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet daa59a18ae624bc8975bee2c460895a7 (table default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]) successfully deleted
I20260808 13:35:48.283883 6026 tablet_bootstrap.cc:492] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:48.284058 5325 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:46324 (request call id 44) took 2076 ms. Trace:
I20260808 13:35:48.284548 5325 rpcz_store.cc:276] 0808 13:35:46.167033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:46.167200 (+ 167us) service_pool.cc:227] Handling call
0808 13:35:47.407250 (+1240050us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b5400140354. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c27a25268 00007f5c28a116cd 00007f5c272b9e04 00007f5c272b9d92 00007f5c272b9cc1 00007f5c272b9c3c 00007f5c272b8ef2 00007f5c257da94c 00007f5c257d9d86 00007f5c257d972f
0808 13:35:47.571083 (+163833us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b5400013954. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c27a25268 00007f5c28a116cd 00007f5c272b9e04 00007f5c272b9d92 00007f5c272b9cc1 00007f5c272b9c3c 00007f5c272b8ef2 00007f5c257da94c 00007f5c257d9d86 00007f5c257d972f
0808 13:35:47.775151 (+204068us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54000a4454. stack: 00007f5c20aa4033 00007f5c20aa3f30 00007f5c20625534 00000000004db2b4 00000000004db249 00007f5c279b0471 00007f5c27a25268 00007f5c28a116cd 00007f5c272b9e04 00007f5c272b9d92 00007f5c272b9cc1 00007f5c272b9c3c 00007f5c272b8ef2 00007f5c257da94c 00007f5c257d9d86 00007f5c257d972f
0808 13:35:48.243457 (+468306us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1777720960}
I20260808 13:35:48.293820 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04d3dd028da4e7f86689e819548d0c5" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:48.308322 8379 raft_consensus.cc:482] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.308874 8379 raft_consensus.cc:504] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.309970 8449 raft_consensus.cc:482] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.310648 8449 raft_consensus.cc:504] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.312415 8379 leader_election.cc:288] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:48.314591 8449 leader_election.cc:288] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:48.315804 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37495b29698940edb2924eb612e94bd7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:48.317160 5550 leader_election.cc:341] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.327598 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594afd4174644c218951b9df82360bfe" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:48.328992 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594afd4174644c218951b9df82360bfe" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:48.328737 8477 tablet.cc:1722] T f66b38e83dab4bf1a3d2d1e088a96988 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:48.329658 5640 raft_consensus.cc:2454] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
W20260808 13:35:48.332785 5551 leader_election.cc:341] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:48.333989 5771 leader_election.cc:341] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.335386 5551 leader_election.cc:302] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:48.336144 8450 raft_consensus.cc:2726] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:48.337183 5772 leader_election.cc:341] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 7e8cf06462484c99a860762a82f7ad64
I20260808 13:35:48.341055 6026 tablet_bootstrap.cc:654] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:48.345296 8449 raft_consensus.cc:482] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.345821 8449 raft_consensus.cc:504] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.347492 5772 leader_election.cc:302] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:48.348212 8315 raft_consensus.cc:2781] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:48.348634 8315 raft_consensus.cc:482] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:48.349102 8315 raft_consensus.cc:3037] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:48.349778 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa86f45770d44e7b993b240607ef2fc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:48.347529 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37495b29698940edb2924eb612e94bd7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:48.351480 5550 leader_election.cc:341] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.351951 8449 leader_election.cc:288] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:48.353027 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bed5e6d1aa49a9b14ba86403c985ce" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:48.355402 6033 tablet_bootstrap.cc:654] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:48.355531 8306 ts_tablet_manager.cc:1918] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:48.356245 8306 log.cc:1196] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/86c2b34c95244472beec43643e60ff5f
W20260808 13:35:48.356734 5551 leader_election.cc:341] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.357208 5551 leader_election.cc:302] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:48.357563 8306 ts_tablet_manager.cc:1939] T 86c2b34c95244472beec43643e60ff5f P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:48.357988 8450 raft_consensus.cc:2726] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:48.358155 8419 tablet.cc:1722] T 52deb2e968c04b99a436e04ae78d9969 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:48.266383 8303 ts_tablet_manager.cc:1918] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:48.368952 8303 log.cc:1196] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/196d80d806b0432cacf2ca6aa0da8c6b
I20260808 13:35:48.371750 8303 ts_tablet_manager.cc:1939] T 196d80d806b0432cacf2ca6aa0da8c6b P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:48.378333 8450 raft_consensus.cc:482] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.379057 8450 raft_consensus.cc:504] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.389796 8306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 445) took 2477 ms. Trace:
I20260808 13:35:48.391074 8450 leader_election.cc:288] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:48.403499 5551 leader_election.cc:341] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.406519 5551 leader_election.cc:302] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:48.408331 6026 tablet_bootstrap.cc:492] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:48.408842 8450 raft_consensus.cc:2726] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:48.390301 8306 rpcz_store.cc:276] 0808 13:35:45.912299 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.912445 (+ 146us) service_pool.cc:227] Handling call
0808 13:35:48.027836 (+2115391us) ts_tablet_manager.cc:1145] Deleting tablet 86c2b34c95244472beec43643e60ff5f
0808 13:35:48.027890 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.232020 (+204130us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.355227 (+123207us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.389719 (+ 34492us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":165427,"tablet-delete.queue_time_us":2114591}
I20260808 13:35:48.413959 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 86c2b34c95244472beec43643e60ff5f (table default.Table_2df92438979447cc89b7a5301a13d6a9 [id=40cafe491872437cba9d7294fff5519a]) successfully deleted
I20260808 13:35:48.414233 8306 tablet_replica.cc:330] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:48.416847 8306 raft_consensus.cc:2229] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:48.417606 8306 raft_consensus.cc:2258] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:48.420650 8306 ts_tablet_manager.cc:1905] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:48.408846 6026 ts_tablet_manager.cc:1397] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.125s user 0.015s sys 0.002s
I20260808 13:35:48.307552 8490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:48.447139 6026 raft_consensus.cc:348] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.463378 8363 tablet_replica.cc:330] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:48.465122 8363 raft_consensus.cc:2229] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:48.458948 6026 raft_consensus.cc:374] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.466387 6026 raft_consensus.cc:729] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:48.466475 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa86f45770d44e7b993b240607ef2fc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:48.467123 6026 consensus_queue.cc:260] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.467386 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:48.468961 6026 ts_tablet_manager.cc:1428] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.041s user 0.006s sys 0.001s
I20260808 13:35:48.470077 6026 tablet_bootstrap.cc:492] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:48.480661 6026 tablet_bootstrap.cc:654] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:48.487249 8371 raft_consensus.cc:482] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.487928 8371 raft_consensus.cc:504] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.498492 6026 tablet_bootstrap.cc:492] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:48.498950 6026 ts_tablet_manager.cc:1397] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.029s user 0.012s sys 0.010s
I20260808 13:35:48.500373 8363 raft_consensus.cc:2258] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:48.501750 6046 tablet_bootstrap.cc:492] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:48.502168 6046 ts_tablet_manager.cc:1397] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.495s user 0.011s sys 0.006s
W20260808 13:35:48.505066 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 204240us
I20260808 13:35:48.512985 6026 raft_consensus.cc:348] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.513842 6026 raft_consensus.cc:374] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.514107 6026 raft_consensus.cc:729] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
W20260808 13:35:48.515136 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.082s user 0.002s sys 0.001s
I20260808 13:35:48.514730 6026 consensus_queue.cc:260] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.519654 8363 ts_tablet_manager.cc:1905] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:48.525931 6026 ts_tablet_manager.cc:1428] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.027s user 0.004s sys 0.004s
I20260808 13:35:48.527110 6026 tablet_bootstrap.cc:492] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:48.531030 8371 leader_election.cc:288] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:48.532217 6046 raft_consensus.cc:348] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.533123 6046 raft_consensus.cc:374] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.533470 6046 raft_consensus.cc:729] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:48.534071 6026 tablet_bootstrap.cc:654] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:48.534209 6046 consensus_queue.cc:260] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.536514 6046 ts_tablet_manager.cc:1428] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.034s user 0.003s sys 0.004s
I20260808 13:35:48.537824 6046 tablet_bootstrap.cc:492] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:48.383401 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:48.384167 8409 tablet.cc:1722] T 52deb2e968c04b99a436e04ae78d9969 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:48.541379 6026 tablet_bootstrap.cc:492] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:48.541771 6026 ts_tablet_manager.cc:1397] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.003s
I20260808 13:35:48.544799 6026 raft_consensus.cc:348] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.545392 6026 raft_consensus.cc:374] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.545611 6026 raft_consensus.cc:729] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:48.546126 6026 consensus_queue.cc:260] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.549810 6026 ts_tablet_manager.cc:1428] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 13:35:48.550735 6026 tablet_bootstrap.cc:492] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:48.555740 6026 tablet_bootstrap.cc:654] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:48.557101 8303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 450) took 2744 ms. Trace:
I20260808 13:35:48.557252 8306 ts_tablet_manager.cc:1918] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:48.557628 8306 log.cc:1196] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/5c0b76c4fdd94a6e9d9bb7709ff1b4be
I20260808 13:35:48.558306 8306 ts_tablet_manager.cc:1939] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:48.566296 6046 tablet_bootstrap.cc:654] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:48.442337 8490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:48.567374 6026 tablet_bootstrap.cc:492] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:48.595182 6026 ts_tablet_manager.cc:1397] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.045s user 0.015s sys 0.000s
I20260808 13:35:48.597595 6026 raft_consensus.cc:348] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.598256 6026 raft_consensus.cc:374] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.598512 6026 raft_consensus.cc:729] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:48.599191 6026 consensus_queue.cc:260] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.601205 6026 ts_tablet_manager.cc:1428] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 13:35:48.601166 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 489 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:48.602018 6026 tablet_bootstrap.cc:492] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:48.603437 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 489 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:48.603005 8490 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:48.605274 7531 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:48.560475 8306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 446) took 2646 ms. Trace:
I20260808 13:35:48.605896 8306 rpcz_store.cc:276] 0808 13:35:45.913735 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.913834 (+ 99us) service_pool.cc:227] Handling call
0808 13:35:48.413940 (+2500106us) ts_tablet_manager.cc:1145] Deleting tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be
0808 13:35:48.414028 (+ 88us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.456513 (+ 42485us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.557017 (+100504us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.560409 (+ 3392us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2499500}
I20260808 13:35:48.607707 8306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 459) took 1478 ms. Trace:
I20260808 13:35:48.608038 8306 rpcz_store.cc:276] 0808 13:35:47.129367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.129552 (+ 185us) service_pool.cc:227] Handling call
0808 13:35:48.607341 (+1477789us) ts_tablet_manager.cc:1145] Deleting tablet 16736500f94c43d3a3163f61b7606aad
0808 13:35:48.607389 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.607653 (+ 264us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1476831}
I20260808 13:35:48.480646 8370 raft_consensus.cc:504] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.611773 8490 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:48.616688 6026 tablet_bootstrap.cc:654] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:48.633208 8363 ts_tablet_manager.cc:1918] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:48.633664 8363 log.cc:1196] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/196d80d806b0432cacf2ca6aa0da8c6b
W20260808 13:35:48.639460 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.121s user 0.001s sys 0.001s
I20260808 13:35:48.644124 8363 ts_tablet_manager.cc:1939] T 196d80d806b0432cacf2ca6aa0da8c6b P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:48.648202 8363 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 459) took 2778 ms. Trace:
W20260808 13:35:48.648355 4784 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 122360us
I20260808 13:35:48.664472 6046 tablet_bootstrap.cc:492] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:48.665171 6046 ts_tablet_manager.cc:1397] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.128s user 0.014s sys 0.001s
I20260808 13:35:48.669773 6046 raft_consensus.cc:348] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.674743 6046 raft_consensus.cc:374] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.675171 6046 raft_consensus.cc:729] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:48.670562 8363 rpcz_store.cc:276] 0808 13:35:45.870215 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.870571 (+ 356us) service_pool.cc:227] Handling call
0808 13:35:48.463063 (+2592492us) ts_tablet_manager.cc:1145] Deleting tablet 196d80d806b0432cacf2ca6aa0da8c6b
0808 13:35:48.463136 (+ 73us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.588249 (+125113us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.632927 (+ 44678us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.648072 (+ 15145us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":329,"tablet-delete.queue_time_us":2575126}
I20260808 13:35:48.676541 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 196d80d806b0432cacf2ca6aa0da8c6b (table default.Table_f212f0d1d7294b4f80606ba37c7f19f3 [id=8039834d1c7f4cb6894534c8284b27c1]) successfully deleted
I20260808 13:35:48.676996 8363 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 464) took 2669 ms. Trace:
I20260808 13:35:48.677357 8363 rpcz_store.cc:276] 0808 13:35:46.007360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:46.007517 (+ 157us) service_pool.cc:227] Handling call
0808 13:35:48.676620 (+2669103us) ts_tablet_manager.cc:1145] Deleting tablet 553a01330cfe4dd38a61860dc943fbbc
0808 13:35:48.676666 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.676941 (+ 275us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2665034}
W20260808 13:35:48.678403 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 553a01330cfe4dd38a61860dc943fbbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 553a01330cfe4dd38a61860dc943fbbc already in progress: creating tablet
I20260808 13:35:48.678506 6046 consensus_queue.cc:260] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.681200 6046 ts_tablet_manager.cc:1428] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.016s user 0.003s sys 0.004s
I20260808 13:35:48.505350 8418 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:54830 (request call id 468) took 1340 ms. Trace:
I20260808 13:35:48.532284 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff163c36c6d42238cce2396102f9a68" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:48.557569 8303 rpcz_store.cc:276] 0808 13:35:45.812735 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.812853 (+ 118us) service_pool.cc:227] Handling call
0808 13:35:47.462457 (+1649604us) ts_tablet_manager.cc:1145] Deleting tablet 196d80d806b0432cacf2ca6aa0da8c6b
0808 13:35:47.462577 (+ 120us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:47.966434 (+503857us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.266055 (+299621us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.557033 (+290978us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":123925,"tablet-delete.queue_time_us":1648511}
I20260808 13:35:48.688421 5880 raft_consensus.cc:2454] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:48.689886 5283 leader_election.cc:302] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:48.690690 8491 raft_consensus.cc:2781] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:48.691026 8491 raft_consensus.cc:482] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:48.691318 8491 raft_consensus.cc:3037] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:48.686188 8418 rpcz_store.cc:276] 0808 13:35:47.165090 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.215538 (+ 50448us) service_pool.cc:227] Handling call
0808 13:35:48.505236 (+1289698us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:47.235154 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0808 13:35:47.235901 (+ 747us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0808 13:35:47.235991 (+ 90us) alter_schema_op.cc:65] Acquired schema lock
0808 13:35:47.236209 (+ 218us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0808 13:35:47.236349 (+ 140us) alter_schema_op.cc:126] START. Timestamp: 1786196147235330
0808 13:35:47.236410 (+ 61us) op_driver.cc:331] REPLICATION: starting
0808 13:35:47.237399 (+ 989us) log.cc:842] Serialized 173 byte log entry
0808 13:35:47.629334 (+391935us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b1800106a80. stack: 00007f19df247033 00007f19df246f30 00007f19dedc8534 00000000004db2b4 00000000004db249 00007f19e80bf8b1 00007f19e80cc661 00007f19df279638 00007f19df27cb11 00007f19df27cac9 00007f19df27ca59 00007f19df27ca21 00007f19df27bd1d 0000000000580784 00000000005805b9 00007f19df25bdc6
0808 13:35:47.830957 (+201623us) op_driver.cc:444] REPLICATION: finished
0808 13:35:47.831703 (+ 746us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0808 13:35:47.920533 (+ 88830us) log.cc:842] Serialized 5 byte log entry
0808 13:35:48.276161 (+355628us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:48.277689 (+ 1528us) log.cc:842] Serialized 14 byte log entry
0808 13:35:48.504950 (+227261us) alter_schema_op.cc:71] Released schema lock
0808 13:35:48.504996 (+ 46us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":472,"prepare.queue_time_us":268,"prepare.run_cpu_time_us":9649,"prepare.run_wall_time_us":201177,"replication_time_us":594408,"spinlock_wait_cycles":65413888,"thread_start_us":99272,"threads_started":1,"wal-append.queue_time_us":233527}]]}
I20260808 13:35:48.704161 8370 leader_election.cc:288] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:48.706578 6046 tablet_bootstrap.cc:492] T 2d8e4af746774747a70d425068e8dd63 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:48.735906 8491 raft_consensus.cc:504] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.741226 8491 leader_election.cc:288] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:48.743958 8363 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 477) took 1307 ms. Trace:
I20260808 13:35:48.744417 8363 rpcz_store.cc:276] 0808 13:35:47.436997 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.443942 (+ 6945us) service_pool.cc:227] Handling call
0808 13:35:48.681766 (+1237824us) ts_tablet_manager.cc:1145] Deleting tablet 16736500f94c43d3a3163f61b7606aad
0808 13:35:48.743563 (+ 61797us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.743894 (+ 331us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1237221}
W20260808 13:35:48.745772 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 16736500f94c43d3a3163f61b7606aad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16736500f94c43d3a3163f61b7606aad already in progress: creating tablet
I20260808 13:35:48.747012 8363 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 479) took 1114 ms. Trace:
I20260808 13:35:48.747367 8363 rpcz_store.cc:276] 0808 13:35:47.632888 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.633025 (+ 137us) service_pool.cc:227] Handling call
0808 13:35:48.746650 (+1113625us) ts_tablet_manager.cc:1145] Deleting tablet 26ce3623d547476684782fd395fb8fcf
0808 13:35:48.746697 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.746962 (+ 265us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1017272}
W20260808 13:35:48.749209 4820 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 26ce3623d547476684782fd395fb8fcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26ce3623d547476684782fd395fb8fcf already in progress: creating tablet
I20260808 13:35:48.751523 6046 tablet_bootstrap.cc:654] T 2d8e4af746774747a70d425068e8dd63 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:48.754208 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff163c36c6d42238cce2396102f9a68" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:48.755045 5880 raft_consensus.cc:3037] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:48.760232 6033 tablet_bootstrap.cc:492] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:48.760659 6033 ts_tablet_manager.cc:1397] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.596s user 0.008s sys 0.010s
I20260808 13:35:48.764022 6033 raft_consensus.cc:348] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.764758 6033 raft_consensus.cc:374] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.765039 6033 raft_consensus.cc:729] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:48.765661 6033 consensus_queue.cc:260] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.768769 6033 ts_tablet_manager.cc:1428] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260808 13:35:48.771493 6046 tablet_bootstrap.cc:492] T 2d8e4af746774747a70d425068e8dd63 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:48.771939 6046 ts_tablet_manager.cc:1397] T 2d8e4af746774747a70d425068e8dd63 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.066s user 0.009s sys 0.000s
I20260808 13:35:48.780217 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 196d80d806b0432cacf2ca6aa0da8c6b (table default.Table_f212f0d1d7294b4f80606ba37c7f19f3 [id=8039834d1c7f4cb6894534c8284b27c1]) successfully deleted
W20260808 13:35:48.806560 5551 leader_election.cc:341] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.807502 5551 leader_election.cc:302] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:48.810063 8515 raft_consensus.cc:482] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.811933 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8cf06462484c99a860762a82f7ad64" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:48.811180 8515 raft_consensus.cc:504] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.812769 5377 raft_consensus.cc:3037] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:48.814905 8449 raft_consensus.cc:482] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.815600 8449 raft_consensus.cc:504] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.817416 8356 raft_consensus.cc:482] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.817885 8449 leader_election.cc:288] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:48.817982 8356 raft_consensus.cc:504] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.820390 8450 raft_consensus.cc:2726] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:48.774289 6046 raft_consensus.cc:348] T 2d8e4af746774747a70d425068e8dd63 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.822826 6046 raft_consensus.cc:374] T 2d8e4af746774747a70d425068e8dd63 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.823243 6046 raft_consensus.cc:729] T 2d8e4af746774747a70d425068e8dd63 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:48.783339 8303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 441) took 3037 ms. Trace:
I20260808 13:35:48.823740 8303 rpcz_store.cc:276] 0808 13:35:45.715734 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.898241 (+182507us) service_pool.cc:227] Handling call
0808 13:35:48.698679 (+2800438us) ts_tablet_manager.cc:1145] Deleting tablet 11cd8f12cda94c4eb9a848f002a4020b
0808 13:35:48.700433 (+ 1754us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.753358 (+ 52925us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2788076}
I20260808 13:35:48.824018 6046 consensus_queue.cc:260] T 2d8e4af746774747a70d425068e8dd63 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.825472 8303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 453) took 2951 ms. Trace:
I20260808 13:35:48.825843 8303 rpcz_store.cc:276] 0808 13:35:45.873835 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.874117 (+ 282us) service_pool.cc:227] Handling call
0808 13:35:48.825062 (+2950945us) ts_tablet_manager.cc:1145] Deleting tablet 553a01330cfe4dd38a61860dc943fbbc
0808 13:35:48.825108 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.825414 (+ 306us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2896370}
I20260808 13:35:48.826691 6046 ts_tablet_manager.cc:1428] T 2d8e4af746774747a70d425068e8dd63 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.054s user 0.008s sys 0.000s
W20260808 13:35:48.827080 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 11cd8f12cda94c4eb9a848f002a4020b because the tablet was not found. No further retry: Not found: Tablet not found: 11cd8f12cda94c4eb9a848f002a4020b
I20260808 13:35:48.827293 8303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 467) took 1689 ms. Trace:
I20260808 13:35:48.827587 8303 rpcz_store.cc:276] 0808 13:35:47.138307 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.138722 (+ 415us) service_pool.cc:227] Handling call
0808 13:35:48.826879 (+1688157us) ts_tablet_manager.cc:1145] Deleting tablet 16736500f94c43d3a3163f61b7606aad
0808 13:35:48.826920 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.827243 (+ 323us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1687553}
W20260808 13:35:48.828193 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 553a01330cfe4dd38a61860dc943fbbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 553a01330cfe4dd38a61860dc943fbbc already in progress: creating tablet
I20260808 13:35:48.828766 8303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 469) took 1201 ms. Trace:
I20260808 13:35:48.828830 8515 leader_election.cc:288] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:48.829249 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 16736500f94c43d3a3163f61b7606aad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16736500f94c43d3a3163f61b7606aad already in progress: creating tablet
I20260808 13:35:48.829078 8303 rpcz_store.cc:276] 0808 13:35:47.627771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.627976 (+ 205us) service_pool.cc:227] Handling call
0808 13:35:48.828428 (+1200452us) ts_tablet_manager.cc:1145] Deleting tablet 26ce3623d547476684782fd395fb8fcf
0808 13:35:48.828467 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:48.828716 (+ 249us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1199566}
I20260808 13:35:48.830646 6033 tablet_bootstrap.cc:492] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:48.827925 8356 leader_election.cc:288] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:48.514496 8315 raft_consensus.cc:504] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:48.836920 4819 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 26ce3623d547476684782fd395fb8fcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26ce3623d547476684782fd395fb8fcf already in progress: creating tablet
I20260808 13:35:48.841084 6046 tablet_bootstrap.cc:492] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:48.849011 5551 leader_election.cc:341] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.826452 5377 raft_consensus.cc:2454] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 2.
W20260808 13:35:48.850524 5550 leader_election.cc:341] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.850946 5550 leader_election.cc:302] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:48.851837 8524 raft_consensus.cc:2726] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:48.856887 5880 raft_consensus.cc:2454] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:48.858070 5283 leader_election.cc:302] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:48.858901 8491 raft_consensus.cc:2781] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:48.859377 8491 raft_consensus.cc:686] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:48.859885 6026 tablet_bootstrap.cc:492] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:48.860587 6026 ts_tablet_manager.cc:1397] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.259s user 0.012s sys 0.005s
I20260808 13:35:48.860214 8491 consensus_queue.cc:237] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.871626 8315 leader_election.cc:288] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:48.870297 6026 raft_consensus.cc:348] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.878963 6026 raft_consensus.cc:374] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:48.879341 6026 raft_consensus.cc:729] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:48.880062 6026 consensus_queue.cc:260] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:48.882939 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 228ad48e495b45a29e96d427b7098497 (table default.Table_f1ad878382f04d038b6b08429d6b9db9 [id=d84314027def49d380b8c13a0431f39e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 228ad48e495b45a29e96d427b7098497 consensus state has no leader
I20260808 13:35:48.883085 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594afd4174644c218951b9df82360bfe" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:48.883991 5629 raft_consensus.cc:3037] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:48.889382 5772 leader_election.cc:341] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 7e8cf06462484c99a860762a82f7ad64
W20260808 13:35:48.890019 5772 consensus_peers.cc:487] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 86c2b34c95244472beec43643e60ff5f. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:48.890146 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e36d643850411498c23699835f55a6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:48.890729 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756b5f24ed94a9fbe2a43efc7b8d40d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:48.891278 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b2cce690cd462aa58ceee2bd28c638" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:48.891499 8370 raft_consensus.cc:482] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.891937 8370 raft_consensus.cc:504] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.893918 8370 leader_election.cc:288] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:48.897218 6046 tablet_bootstrap.cc:654] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:48.902645 5551 leader_election.cc:341] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:48.903234 5551 leader_election.cc:341] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:48.903708 5551 leader_election.cc:341] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.907367 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64888f0d27c94ee7b25c0dda57e766f6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:48.907725 6026 ts_tablet_manager.cc:1428] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.047s user 0.007s sys 0.001s
I20260808 13:35:48.908232 5644 raft_consensus.cc:2454] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:48.909062 6026 tablet_bootstrap.cc:492] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:48.923741 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.136s user 0.001s sys 0.000s
I20260808 13:35:48.927891 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756b5f24ed94a9fbe2a43efc7b8d40d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:48.931869 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b2cce690cd462aa58ceee2bd28c638" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:48.932647 5772 leader_election.cc:302] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:48.933916 8512 raft_consensus.cc:2781] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:48.934232 8512 raft_consensus.cc:482] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:48.934510 8512 raft_consensus.cc:3037] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:48.935487 6026 tablet_bootstrap.cc:654] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:48.937176 5629 raft_consensus.cc:2454] T 594afd4174644c218951b9df82360bfe P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
W20260808 13:35:48.945261 5550 leader_election.cc:341] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.945410 5772 leader_election.cc:302] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:48.945649 5550 leader_election.cc:302] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:48.946866 8370 raft_consensus.cc:2781] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:48.946866 8524 raft_consensus.cc:2726] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:48.947316 8370 raft_consensus.cc:686] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
W20260808 13:35:48.948066 5550 leader_election.cc:341] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.948422 5550 leader_election.cc:302] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:48.948076 8370 consensus_queue.cc:237] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.949960 8524 raft_consensus.cc:2726] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:48.963153 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be (table default.Table_d553a5c837764412ab01800bea0017c2 [id=81ee34de9d5c494fb2bd74557a174c14]) successfully deleted
I20260808 13:35:48.969223 5377 raft_consensus.cc:3037] T 2d8e4af746774747a70d425068e8dd63 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:48.970861 8371 raft_consensus.cc:482] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.971442 8371 raft_consensus.cc:504] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:48.972970 8524 raft_consensus.cc:482] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:48.973361 8524 raft_consensus.cc:504] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:48.978294 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba5fa465ec04bbb801a821a71a8f87d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:48.979367 5551 leader_election.cc:341] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:48.980072 8371 leader_election.cc:288] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:48.980260 8524 leader_election.cc:288] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:48.981292 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a27517560d64a3b8da2cf55980e18a8" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:48.981910 5869 raft_consensus.cc:2454] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:48.982198 8524 consensus_queue.cc:1035] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:48.983261 5283 leader_election.cc:302] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:48.984205 8371 raft_consensus.cc:2781] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:35:48.984555 8371 raft_consensus.cc:482] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:48.984887 8371 raft_consensus.cc:3037] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Advancing to term 2
W20260808 13:35:49.016362 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 16736500f94c43d3a3163f61b7606aad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16736500f94c43d3a3163f61b7606aad already in progress: creating tablet
W20260808 13:35:49.017664 4821 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 26ce3623d547476684782fd395fb8fcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26ce3623d547476684782fd395fb8fcf already in progress: creating tablet
I20260808 13:35:49.049750 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba5fa465ec04bbb801a821a71a8f87d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:49.052484 5550 leader_election.cc:341] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.052925 5550 leader_election.cc:302] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:49.054522 8514 raft_consensus.cc:2726] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:49.068109 6033 tablet_bootstrap.cc:654] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:49.071022 5771 leader_election.cc:302] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:49.074424 8510 raft_consensus.cc:2781] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:35:49.079236 8510 raft_consensus.cc:686] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:49.080163 8510 consensus_queue.cc:237] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.086066 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594afd4174644c218951b9df82360bfe" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:49.086221 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64888f0d27c94ee7b25c0dda57e766f6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:49.086781 5377 raft_consensus.cc:3037] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:49.078249 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e36d643850411498c23699835f55a6" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:49.094189 5771 leader_election.cc:341] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.094719 5377 raft_consensus.cc:2454] T 594afd4174644c218951b9df82360bfe P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:49.110625 8477 tablet.cc:1722] T 52deb2e968c04b99a436e04ae78d9969 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:49.113099 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff163c36c6d42238cce2396102f9a68" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:49.113875 5644 raft_consensus.cc:2454] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:48.983942 8512 raft_consensus.cc:504] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.125178 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64888f0d27c94ee7b25c0dda57e766f6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:49.128767 5644 raft_consensus.cc:3037] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:49.126259 8518 raft_consensus.cc:482] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:49.129700 8518 raft_consensus.cc:504] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:49.135689 6046 tablet_bootstrap.cc:492] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:49.136248 6046 ts_tablet_manager.cc:1397] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.309s user 0.040s sys 0.025s
I20260808 13:35:49.138785 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa049ef916b44458be43ec4e2ccaf1c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:49.141995 5551 leader_election.cc:341] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.147842 8518 leader_election.cc:288] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.151602 5644 raft_consensus.cc:2454] T 64888f0d27c94ee7b25c0dda57e766f6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:49.154160 5772 leader_election.cc:302] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:49.149236 6046 raft_consensus.cc:348] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.154892 6046 raft_consensus.cc:374] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:49.155169 6046 raft_consensus.cc:729] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:49.152379 6026 tablet_bootstrap.cc:492] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:49.156049 6026 ts_tablet_manager.cc:1397] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.247s user 0.037s sys 0.023s
I20260808 13:35:49.155874 6046 consensus_queue.cc:260] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.158490 6026 raft_consensus.cc:348] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.159164 6026 raft_consensus.cc:374] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:49.159401 6026 raft_consensus.cc:729] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:49.162621 6026 consensus_queue.cc:260] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.164356 6026 ts_tablet_manager.cc:1428] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260808 13:35:49.167474 6046 ts_tablet_manager.cc:1428] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260808 13:35:49.168419 6046 tablet_bootstrap.cc:492] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:49.169716 5550 leader_election.cc:341] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.170147 5550 leader_election.cc:302] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:49.171629 8518 raft_consensus.cc:2726] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:49.182050 6046 tablet_bootstrap.cc:654] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:49.124123 8512 leader_election.cc:288] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:49.184054 8371 raft_consensus.cc:504] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.187064 8371 leader_election.cc:288] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.188073 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff163c36c6d42238cce2396102f9a68" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:49.188759 5640 raft_consensus.cc:3037] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:49.197857 5640 raft_consensus.cc:2454] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:49.199016 8541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:49.202239 8541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:49.205118 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 489 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:49.223212 8541 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:49.224933 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 4 candidate_status { last_received { term: 3 index: 489 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:49.226498 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:49.227852 8541 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:35:49.261281 6026 tablet_bootstrap.cc:492] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:49.262642 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 167350us
I20260808 13:35:49.267650 8524 raft_consensus.cc:482] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:49.268141 8524 raft_consensus.cc:504] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:49.270370 8524 leader_election.cc:288] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.272372 6026 tablet_bootstrap.cc:654] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:49.281637 6033 tablet_bootstrap.cc:492] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
W20260808 13:35:49.271580 5285 consensus_peers.cc:487] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 851a0b374a854efd87dd66e39b9df2d5. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:49.282748 6033 ts_tablet_manager.cc:1397] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.452s user 0.010s sys 0.000s
I20260808 13:35:49.284230 8465 consensus_queue.cc:1035] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:49.272204 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3e9c6770fd402290b45eed85de6f6f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:49.276527 6046 tablet_bootstrap.cc:492] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:49.285825 6046 ts_tablet_manager.cc:1397] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.118s user 0.016s sys 0.017s
W20260808 13:35:49.288336 5551 leader_election.cc:341] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.288455 6026 tablet_bootstrap.cc:492] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:49.288955 6026 ts_tablet_manager.cc:1397] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.003s
I20260808 13:35:49.288585 6046 raft_consensus.cc:348] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.289302 6046 raft_consensus.cc:374] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:49.289578 6046 raft_consensus.cc:729] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:49.290196 6046 consensus_queue.cc:260] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:49.291199 5285 leader_election.cc:341] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 3a27517560d64a3b8da2cf55980e18a8
I20260808 13:35:49.291213 6026 raft_consensus.cc:348] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:49.291834 6026 raft_consensus.cc:374] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:49.292064 6026 raft_consensus.cc:729] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:49.292606 6026 consensus_queue.cc:260] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:49.293488 5285 leader_election.cc:341] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 3a27517560d64a3b8da2cf55980e18a8
I20260808 13:35:49.295249 8491 raft_consensus.cc:482] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:49.295639 8491 raft_consensus.cc:504] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.297876 8491 leader_election.cc:288] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.300662 6046 ts_tablet_manager.cc:1428] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260808 13:35:49.301582 6046 tablet_bootstrap.cc:492] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:49.305589 6026 ts_tablet_manager.cc:1428] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260808 13:35:49.319061 8524 raft_consensus.cc:482] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:49.319562 8524 raft_consensus.cc:504] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:49.321923 8524 leader_election.cc:288] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.323596 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c41483119a442eb46816aa34f7f319" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:49.324020 6026 tablet_bootstrap.cc:492] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:49.324169 6046 tablet_bootstrap.cc:654] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:49.324870 5551 leader_election.cc:341] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.340163 6026 tablet_bootstrap.cc:654] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:49.361228 6046 tablet_bootstrap.cc:492] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:49.361660 6046 ts_tablet_manager.cc:1397] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.060s user 0.006s sys 0.005s
I20260808 13:35:49.363848 6046 raft_consensus.cc:348] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.364531 6046 raft_consensus.cc:374] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:49.364790 6046 raft_consensus.cc:729] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:35:49.366885 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 212791us
I20260808 13:35:49.265568 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa049ef916b44458be43ec4e2ccaf1c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:49.377252 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c41483119a442eb46816aa34f7f319" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:49.378705 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3e9c6770fd402290b45eed85de6f6f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:49.378700 6046 consensus_queue.cc:260] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:49.383384 5550 leader_election.cc:341] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.383848 5550 leader_election.cc:302] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:49.386061 6046 ts_tablet_manager.cc:1428] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
W20260808 13:35:49.391287 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 125055us
I20260808 13:35:49.392920 8491 raft_consensus.cc:482] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:49.393379 8491 raft_consensus.cc:504] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.395267 8491 leader_election.cc:288] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.395969 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64888f0d27c94ee7b25c0dda57e766f6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:49.396528 5379 raft_consensus.cc:3037] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:49.405726 5379 raft_consensus.cc:2454] T 64888f0d27c94ee7b25c0dda57e766f6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:49.413787 8552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:49.384856 8323 raft_consensus.cc:2781] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:49.414548 8552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:49.415388 6033 raft_consensus.cc:348] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.416118 6033 raft_consensus.cc:374] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:49.416375 6033 raft_consensus.cc:729] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:49.417012 6033 consensus_queue.cc:260] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.419413 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 594 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:49.421994 8552 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:35:49.426959 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c2b34c95244472beec43643e60ff5f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:49.429132 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa86f45770d44e7b993b240607ef2fc" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:49.425659 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a27517560d64a3b8da2cf55980e18a8" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:49.431416 5872 raft_consensus.cc:3037] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:49.444523 8323 raft_consensus.cc:686] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:49.445406 8323 consensus_queue.cc:237] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:49.481810 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 98461.7us
W20260808 13:35:49.482550 5550 leader_election.cc:341] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.482932 5550 leader_election.cc:302] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:49.484098 5550 leader_election.cc:341] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.485052 8524 raft_consensus.cc:2726] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:49.485610 8524 raft_consensus.cc:2726] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:49.486124 8524 consensus_queue.cc:1035] T 2d8e4af746774747a70d425068e8dd63 P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:49.498564 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 89944.2us
I20260808 13:35:49.499974 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa86f45770d44e7b993b240607ef2fc" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:49.503717 8512 raft_consensus.cc:482] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:49.504154 8512 raft_consensus.cc:504] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.506107 8512 leader_election.cc:288] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:49.508666 5630 raft_consensus.cc:2454] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:49.484416 5550 leader_election.cc:302] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:49.515815 8450 raft_consensus.cc:2726] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:49.533299 5772 leader_election.cc:341] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 5c0b76c4fdd94a6e9d9bb7709ff1b4be
I20260808 13:35:49.539616 7736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 594 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:35:49.540679 7736 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63c875fc5bcd41a9b1bcf3bd4ae321d7 in term 3.
I20260808 13:35:49.543666 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c
I20260808 13:35:49.544561 8552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260808 13:35:49.544922 8552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3e672d7386144b299688abea9c64854c)
I20260808 13:35:49.545269 8552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 3 FOLLOWER]: Advancing to term 4
I20260808 13:35:49.553102 8552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:49.555680 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 594 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c"
I20260808 13:35:49.557510 8552 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 election: Requested vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:35:49.558516 7736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 594 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5"
I20260808 13:35:49.559113 7736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 3 FOLLOWER]: Advancing to term 4
I20260808 13:35:49.568903 7736 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63c875fc5bcd41a9b1bcf3bd4ae321d7 in term 4.
I20260808 13:35:49.569957 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c
I20260808 13:35:49.570725 8552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 4 FOLLOWER]: Leader election won for term 4
I20260808 13:35:49.573359 8518 raft_consensus.cc:482] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:49.573807 8518 raft_consensus.cc:504] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:49.575709 8518 leader_election.cc:288] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.579473 8552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 4 LEADER]: Becoming Leader. State: Replica: 63c875fc5bcd41a9b1bcf3bd4ae321d7, State: Running, Role: LEADER
I20260808 13:35:49.580421 8552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 594, Committed index: 594, Last appended: 3.594, Last appended by leader: 594, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:49.585825 8524 raft_consensus.cc:482] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 98dd96f425fc4ab3a6a54f2dfaaf9fab)
I20260808 13:35:49.586318 8524 raft_consensus.cc:504] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.588282 8524 leader_election.cc:288] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:49.414371 5283 consensus_peers.cc:487] T a73e98a8b20144508a5a8d451f4bb302 P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:49.595064 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 180812us
I20260808 13:35:49.501425 5377 raft_consensus.cc:3037] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:49.507915 5384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0b76c4fdd94a6e9d9bb7709ff1b4be" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:49.598317 5384 raft_consensus.cc:2454] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:49.600221 8565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c875fc5bcd41a9b1bcf3bd4ae321d7. Latest consensus state: current_term: 4 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:35:49.601487 8565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:35:49.566116 5285 leader_election.cc:302] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:49.603085 8491 raft_consensus.cc:2781] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:49.603422 8491 raft_consensus.cc:482] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:49.603760 8491 raft_consensus.cc:3037] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:49.607434 5771 leader_election.cc:302] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
W20260808 13:35:49.608392 5283 leader_election.cc:341] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.608767 8323 raft_consensus.cc:2781] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:49.609105 8323 raft_consensus.cc:482] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:49.609431 8323 raft_consensus.cc:3037] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:49.577581 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7454bf27bc1a45d2af0d8f897b2f9f97" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:49.589725 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:49.611449 5383 raft_consensus.cc:2379] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 2: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:49.615736 8567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:35:49.617204 8491 raft_consensus.cc:504] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.617331 8323 raft_consensus.cc:504] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:49.628296 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 55088.4us
I20260808 13:35:49.646023 5872 raft_consensus.cc:2454] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 2.
I20260808 13:35:49.647351 5283 leader_election.cc:302] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:49.648144 8465 raft_consensus.cc:2781] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:35:49.648605 8465 raft_consensus.cc:686] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 2 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:49.649367 8465 consensus_queue.cc:237] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.650262 8323 leader_election.cc:288] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:49.653518 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0b76c4fdd94a6e9d9bb7709ff1b4be" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:49.654320 5379 raft_consensus.cc:3037] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:49.659833 5551 leader_election.cc:341] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.663352 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:49.664078 5877 raft_consensus.cc:2454] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 2.
W20260808 13:35:49.673108 5285 leader_election.cc:341] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 86c2b34c95244472beec43643e60ff5f
I20260808 13:35:49.673520 5285 leader_election.cc:302] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:49.692140 5772 leader_election.cc:341] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 5c0b76c4fdd94a6e9d9bb7709ff1b4be
I20260808 13:35:49.694460 8491 leader_election.cc:288] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.709879 5550 leader_election.cc:302] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:49.710810 8518 raft_consensus.cc:2781] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:35:49.711300 8518 raft_consensus.cc:482] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 98dd96f425fc4ab3a6a54f2dfaaf9fab)
I20260808 13:35:49.711685 8518 raft_consensus.cc:3037] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:49.713747 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa86f45770d44e7b993b240607ef2fc" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:49.743467 6033 ts_tablet_manager.cc:1428] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.455s user 0.009s sys 0.005s
I20260808 13:35:49.744513 6033 tablet_bootstrap.cc:492] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:49.748337 8338 raft_consensus.cc:2726] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:49.755947 5283 leader_election.cc:341] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.763942 6046 tablet_bootstrap.cc:492] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:49.766903 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7454bf27bc1a45d2af0d8f897b2f9f97" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:49.770478 6046 tablet_bootstrap.cc:654] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:49.780947 5550 leader_election.cc:341] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.791399 5550 leader_election.cc:302] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:49.792552 6046 tablet_bootstrap.cc:492] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:49.793013 6046 ts_tablet_manager.cc:1397] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.000s
I20260808 13:35:49.795804 8524 raft_consensus.cc:2726] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:49.795557 6046 raft_consensus.cc:348] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.795644 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa86f45770d44e7b993b240607ef2fc" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:49.796159 6046 raft_consensus.cc:374] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:49.796259 5634 raft_consensus.cc:3037] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:49.796383 6046 raft_consensus.cc:729] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:49.799114 6046 consensus_queue.cc:260] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.815611 6046 ts_tablet_manager.cc:1428] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.022s user 0.004s sys 0.003s
I20260808 13:35:49.823823 6046 tablet_bootstrap.cc:492] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:49.864775 8524 raft_consensus.cc:482] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:49.865478 8524 raft_consensus.cc:504] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:49.873131 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e3245c8ba54f69bc329a5b75115127" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:49.874954 6046 tablet_bootstrap.cc:654] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:49.876530 5551 leader_election.cc:341] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.877774 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_114cb957447a4943806bdadc6ca86549 [id=000c41391410419bb1d4881b34c602fe]
I20260808 13:35:49.880062 8524 leader_election.cc:288] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.880362 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_c06c463992cf45db8dacc3018d8bcb8a [id=00bfe57fc39c432690f17b74eab8d1fe]
I20260808 13:35:49.880999 5634 raft_consensus.cc:2454] T 2fa86f45770d44e7b993b240607ef2fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:49.882572 5285 leader_election.cc:302] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:49.883167 6026 tablet_bootstrap.cc:492] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:49.883857 6026 ts_tablet_manager.cc:1397] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.560s user 0.012s sys 0.010s
I20260808 13:35:49.884268 8491 raft_consensus.cc:2781] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:49.887158 8491 raft_consensus.cc:686] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:49.889775 8491 consensus_queue.cc:237] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.899187 6026 raft_consensus.cc:348] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:49.900352 6026 raft_consensus.cc:374] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:49.900635 6026 raft_consensus.cc:729] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:49.901666 8450 raft_consensus.cc:482] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:49.902181 8450 raft_consensus.cc:504] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.904580 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e3245c8ba54f69bc329a5b75115127" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:49.906451 8450 leader_election.cc:288] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:49.907393 5550 leader_election.cc:341] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.907573 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba39706cff65404485959865fff1772a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:49.907967 5550 leader_election.cc:302] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:49.909571 5551 leader_election.cc:341] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.910689 8524 raft_consensus.cc:2726] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:49.911273 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba39706cff65404485959865fff1772a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:49.911653 8465 raft_consensus.cc:482] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:49.912233 8465 raft_consensus.cc:504] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.910260 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_43f5211ad48d49e982e3ac2625080359 [id=0111d9bdd1f045b197b24461bc37070f]
I20260808 13:35:49.915556 8518 raft_consensus.cc:504] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.921985 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b2cce690cd462aa58ceee2bd28c638" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:49.919560 8465 leader_election.cc:288] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.922812 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b2cce690cd462aa58ceee2bd28c638" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:49.928611 5634 raft_consensus.cc:2454] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:49.913918 6026 consensus_queue.cc:260] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:49.929682 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 228ad48e495b45a29e96d427b7098497 (table default.Table_f1ad878382f04d038b6b08429d6b9db9 [id=d84314027def49d380b8c13a0431f39e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 228ad48e495b45a29e96d427b7098497 consensus state has no leader
I20260808 13:35:49.930929 6026 ts_tablet_manager.cc:1428] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.047s user 0.007s sys 0.000s
I20260808 13:35:49.930863 5285 leader_election.cc:302] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:49.931908 8465 raft_consensus.cc:2781] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:49.932235 6026 tablet_bootstrap.cc:492] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:49.932523 8465 raft_consensus.cc:482] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260808 13:35:49.934206 5550 leader_election.cc:341] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.934626 5550 leader_election.cc:302] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:49.934743 8465 raft_consensus.cc:3037] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:49.936900 8580 raft_consensus.cc:2726] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:49.938410 6046 tablet_bootstrap.cc:492] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:49.938334 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:49.939292 6046 ts_tablet_manager.cc:1397] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.116s user 0.014s sys 0.007s
I20260808 13:35:49.939859 5377 raft_consensus.cc:2379] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 2: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
W20260808 13:35:49.946038 5283 leader_election.cc:341] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:49.954468 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_f685324c68e647af9888454398dc7d23 [id=01d0f24951d44f7f86fb55decd5990ab]
I20260808 13:35:49.971362 8338 consensus_queue.cc:1035] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:49.977209 8560 consensus_queue.cc:1035] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:49.981747 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_6b9bc590a69746909348fbcbb7856cdf [id=03f2c8cadea14860aee2cd51574379b6]
I20260808 13:35:49.976389 6046 raft_consensus.cc:348] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.985312 6046 raft_consensus.cc:374] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:49.985612 6046 raft_consensus.cc:729] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:49.975468 8465 raft_consensus.cc:504] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.989492 8465 leader_election.cc:288] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:49.990473 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b2cce690cd462aa58ceee2bd28c638" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:49.991389 5634 raft_consensus.cc:3037] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:49.990053 6046 consensus_queue.cc:260] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:49.994303 6026 tablet_bootstrap.cc:654] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:49.983589 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_c2023a99a0654190adb8ada69a32f740 [id=054085276211478a8982d48b0599a684]
I20260808 13:35:49.998893 8518 leader_election.cc:288] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 2 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:50.004153 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_94d7e54302ae4463a43a530d34d4f264 [id=062b1b88fea4482fa36b7456f6534f4f]
I20260808 13:35:49.997244 8491 consensus_queue.cc:1035] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:50.009356 6046 ts_tablet_manager.cc:1428] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.069s user 0.005s sys 0.003s
I20260808 13:35:50.010304 6046 tablet_bootstrap.cc:492] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:50.021967 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_bf8907064282450eb869d5bb8ec9f1c1 [id=07969203fd174637a09f5b8e83bc7402]
I20260808 13:35:50.028721 8491 raft_consensus.cc:482] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.029245 8491 raft_consensus.cc:504] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.031586 8491 leader_election.cc:288] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:50.033964 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04d3dd028da4e7f86689e819548d0c5" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:50.034538 5640 raft_consensus.cc:2454] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:50.036867 5285 leader_election.cc:302] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:50.037719 8465 raft_consensus.cc:2781] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:50.038185 8465 raft_consensus.cc:482] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:50.038560 8465 raft_consensus.cc:3037] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:50.046365 6046 tablet_bootstrap.cc:654] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:50.056061 5379 raft_consensus.cc:2454] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:50.060248 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_465b28acac534aa0960266bb6c7efd27 [id=087eec2212324876bebfc252a552d074]
I20260808 13:35:50.064808 8465 raft_consensus.cc:504] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.083053 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_811d61f013fe4c59853421dff8a641bb [id=093c441116e34908b83de704bbb58ee5]
I20260808 13:35:50.085841 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04d3dd028da4e7f86689e819548d0c5" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:50.086404 5640 raft_consensus.cc:3037] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:50.086894 8465 leader_election.cc:288] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:50.101114 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:50.112403 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_187b31571225409e855d0b4783e4b74c [id=0c03629a169f469897599b731ca031e2]
I20260808 13:35:50.136353 8370 consensus_queue.cc:1035] T 594afd4174644c218951b9df82360bfe P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:50.139963 7736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Refusing update from remote peer 63c875fc5bcd41a9b1bcf3bd4ae321d7: Log matching property violated. Preceding OpId in replica: term: 3 index: 489. Preceding OpId from leader: term: 4 index: 595. (index mismatch)
I20260808 13:35:50.123217 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_9d2867cdebf84440b7aa21a71faba9c4 [id=0ddf25cf69f449cf8c7516cb84736b3d]
I20260808 13:35:50.144281 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_661e11184b9346b1bbae005034dddd62 [id=0ded1d2205b248c8acf4ba8baac43096]
I20260808 13:35:50.146296 8565 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 595, Last known committed idx: 489, Time since last communication: 0.001s
I20260808 13:35:50.172302 5640 raft_consensus.cc:2454] T a04d3dd028da4e7f86689e819548d0c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:50.170286 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_48ee76308d8846b49515e00b6bf24244 [id=11a917856b1045d1891af36f7a71e26c]
I20260808 13:35:50.179193 5771 leader_election.cc:302] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:50.181010 8553 raft_consensus.cc:2781] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:50.182498 8553 raft_consensus.cc:686] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:50.184087 8553 consensus_queue.cc:237] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.185474 6033 tablet_bootstrap.cc:654] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:50.140409 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b2cce690cd462aa58ceee2bd28c638" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:50.187934 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_4cb4f6347fda4d268fd87065c9b063ea [id=11f8a63441444ac1a7aa7165cbe4b696]
I20260808 13:35:50.191730 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447]
I20260808 13:35:50.197921 4864 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 3 LEADER]: Stepping down as leader of term 3
I20260808 13:35:50.198443 4864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e672d7386144b299688abea9c64854c, State: Running, Role: LEADER
I20260808 13:35:50.199294 5285 leader_election.cc:302] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:50.199358 4864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 594, Committed index: 594, Last appended: 3.595, Last appended by leader: 595, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:50.200198 8465 raft_consensus.cc:2781] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:50.200501 4864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 3 FOLLOWER]: Advancing to term 4
I20260808 13:35:50.201037 8465 raft_consensus.cc:686] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:50.202581 8465 consensus_queue.cc:237] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.206585 6046 tablet_bootstrap.cc:492] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:50.210251 6046 ts_tablet_manager.cc:1397] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.200s user 0.014s sys 0.002s
I20260808 13:35:50.215878 6046 raft_consensus.cc:348] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.218199 6046 raft_consensus.cc:374] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:50.218487 6046 raft_consensus.cc:729] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:50.220744 6046 consensus_queue.cc:260] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.223737 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04d3dd028da4e7f86689e819548d0c5" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:50.224552 6046 ts_tablet_manager.cc:1428] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260808 13:35:50.226786 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04d3dd028da4e7f86689e819548d0c5" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:50.230530 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_a14f7675417240ca88f88da86055bceb [id=16cef8437603476b97a8562edba58a41]
I20260808 13:35:50.233686 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_db41a83ae3f34453af64e73302451c19 [id=1816314c549642e3adc9f39e532ab1ca]
I20260808 13:35:50.236871 5866 raft_consensus.cc:2454] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd987c3dbdb4168beb73cd7e577c7cb in term 2.
I20260808 13:35:50.241804 6046 tablet_bootstrap.cc:492] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:50.247359 8553 consensus_queue.cc:1035] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:50.258181 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_206fd86bd0f44e8e9dc38631edff49ca [id=1a31b0d0c87b446cb7662a856510b67e]
I20260808 13:35:50.269438 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_7fd510a91deb449091ff7da8c644fae1 [id=1b86c1f240214c88b29348f33a11503d]
I20260808 13:35:50.271366 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_8a69b0e5577e4c6f9c76504751d2c1a4 [id=1b8762a3419f47bb9c060294f70d62aa]
I20260808 13:35:50.272920 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_702490ba7901458b954c15c4637424b0 [id=1e6b56dff3614e5fb96523262224c209]
I20260808 13:35:50.275390 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_f10f80e85d6c4f87b43c1a53b888716c [id=20c27e5e63df49a8bcf6faef31940880]
I20260808 13:35:50.277025 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_510c442cafba44a9bf48a292ca01fbd4 [id=212e1314b3c04c4d8dd06d5730fa25cf]
I20260808 13:35:50.277534 6046 tablet_bootstrap.cc:654] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:50.284493 5283 leader_election.cc:341] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:50.285633 5283 leader_election.cc:341] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:50.286823 5283 leader_election.cc:341] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:50.290732 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445]
I20260808 13:35:50.292773 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_0f52af74f5c34a0aa2f84a521c188673 [id=2378817b60944a789ff39c84ebdc8991]
I20260808 13:35:50.299726 6046 tablet_bootstrap.cc:492] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:50.300259 6046 ts_tablet_manager.cc:1397] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.059s user 0.005s sys 0.011s
I20260808 13:35:50.303084 6046 raft_consensus.cc:348] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.303805 6046 raft_consensus.cc:374] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:50.304062 6046 raft_consensus.cc:729] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:50.305164 6046 consensus_queue.cc:260] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.314543 6046 ts_tablet_manager.cc:1428] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.014s user 0.003s sys 0.002s
I20260808 13:35:50.315500 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_dbb6f93863704e4e9c573d989a367368 [id=2571402ea4b7478098e9596eb3235370]
I20260808 13:35:50.317373 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_33484efb632a4860bc2485f6713c84d0 [id=26126642d65b4c8bbacd0e4741ed7eee]
I20260808 13:35:50.319128 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013]
I20260808 13:35:50.321549 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_7e0bc5965bd54d788347d0d41c53208f [id=296621c6dba14fbc8305eca14e4396a3]
I20260808 13:35:50.323143 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_db9615359c8549588ff9f1d557189a0d [id=29ca8ec3c8a14e8fb3db662cbc5b6907]
I20260808 13:35:50.330727 5383 raft_consensus.cc:1261] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Refusing update from remote peer 98dd96f425fc4ab3a6a54f2dfaaf9fab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260808 13:35:50.332086 8553 consensus_queue.cc:1035] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:50.341140 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_2d75fe6a74ad43789e515c67b5167ab3 [id=29db8794594e4221be8974258b5127e0]
I20260808 13:35:50.356310 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_6c7876f6c0ea460d8fa5cd00a7baee97 [id=2d102da12b5240a59c2ab0bcacb2c405]
I20260808 13:35:50.367012 8518 raft_consensus.cc:482] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.367784 8518 raft_consensus.cc:504] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.348285 8450 raft_consensus.cc:482] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.382947 6033 tablet_bootstrap.cc:492] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:50.383486 6033 ts_tablet_manager.cc:1397] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.639s user 0.015s sys 0.000s
I20260808 13:35:50.382975 8450 raft_consensus.cc:504] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.387290 8450 leader_election.cc:288] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:50.387105 6033 raft_consensus.cc:348] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.387928 6033 raft_consensus.cc:374] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:50.388221 6033 raft_consensus.cc:729] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:50.390458 8518 leader_election.cc:288] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:50.388819 6033 consensus_queue.cc:260] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:50.394021 5772 consensus_peers.cc:487] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e8cf06462484c99a860762a82f7ad64. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:50.396313 8560 consensus_queue.cc:1035] T 64888f0d27c94ee7b25c0dda57e766f6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:50.398828 6033 ts_tablet_manager.cc:1428] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20260808 13:35:50.401656 8560 raft_consensus.cc:482] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 946c9d8462644deba5b45fa724f1923c)
I20260808 13:35:50.402549 8560 raft_consensus.cc:504] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.407945 8560 leader_election.cc:288] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:50.416492 4864 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Refusing update from remote peer 63c875fc5bcd41a9b1bcf3bd4ae321d7: Log matching property violated. Preceding OpId in replica: term: 3 index: 595. Preceding OpId from leader: term: 4 index: 595. (term mismatch)
I20260808 13:35:50.416932 4864 pending_rounds.cc:76] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Aborting all ops after (but not including) 594
I20260808 13:35:50.417191 4864 pending_rounds.cc:97] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Aborting uncommitted WRITE_OP operation due to leader change: 3.595
E20260808 13:35:50.424554 5209 catalog_manager.cc:816] Error processing pending assignments: Aborted: error persisting updated tablet metadata: Op aborted by new leader
I20260808 13:35:50.453799 6033 tablet_bootstrap.cc:492] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:50.401787 5634 raft_consensus.cc:2454] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:50.460357 6033 tablet_bootstrap.cc:654] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:50.474112 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 313453us
I20260808 13:35:50.475857 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356e15bbcd26409a9d6302599816e15f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:50.476917 5550 leader_election.cc:302] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:50.477531 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d68ab73bae4dd9af46e907b1fd198c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:50.483585 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_eeada3d5a8ba4dc4b96d74955c23f2bc [id=2fa9c2e807544cc8916c41200a0691cf]
I20260808 13:35:50.485301 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_5d7feb173847448393c48160e861f0c1 [id=308f2ec39c62416ab0d396aad04351fd]
I20260808 13:35:50.486089 6033 tablet_bootstrap.cc:492] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:50.486553 6033 ts_tablet_manager.cc:1397] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.033s user 0.014s sys 0.000s
I20260808 13:35:50.489037 6033 raft_consensus.cc:348] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.489692 6033 raft_consensus.cc:374] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:50.489954 6033 raft_consensus.cc:729] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:50.492961 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
W20260808 13:35:50.500797 5550 leader_election.cc:341] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:50.501580 5550 leader_election.cc:341] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:50.504386 8565 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 595, Last known committed idx: 594, Time since last communication: 0.001s
I20260808 13:35:50.490557 6033 consensus_queue.cc:260] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:50.539604 5772 leader_election.cc:341] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: daa59a18ae624bc8975bee2c460895a7
I20260808 13:35:50.549799 6033 ts_tablet_manager.cc:1428] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.063s user 0.006s sys 0.000s
I20260808 13:35:50.550709 6033 tablet_bootstrap.cc:492] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:50.505049 8524 raft_consensus.cc:2781] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:35:50.506999 8580 raft_consensus.cc:482] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.531177 6026 tablet_bootstrap.cc:492] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:50.563975 6026 ts_tablet_manager.cc:1397] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.632s user 0.020s sys 0.007s
W20260808 13:35:50.564661 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 63107.2us
I20260808 13:35:50.568316 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356e15bbcd26409a9d6302599816e15f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:50.571609 6046 tablet_bootstrap.cc:492] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:50.573588 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 174032us
I20260808 13:35:50.598037 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_efd769b05e1142088d56528e943072ed [id=31c20a71a6144b238a2da27b7ea21863]
I20260808 13:35:50.600124 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_d2c33a8a16d34fb0a22e445a3c1be52a [id=31d10fd1087f45349072cd842a68c4a0]
I20260808 13:35:50.604643 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:50.571650 8614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c875fc5bcd41a9b1bcf3bd4ae321d7. Latest consensus state: current_term: 4 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:35:50.606137 8614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:35:50.612875 8614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:35:50.613813 8614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:35:50.615607 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_3e5cfb8683d745f08a7738ed4d2cd13a [id=31e622ce08da4164936a264736f75833]
I20260808 13:35:50.619158 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_e024f29e457447139994e4d9c43a41ce [id=334643ef0e864267b0f17674d4f1c06c]
I20260808 13:35:50.619640 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 538d9a883e9d46668fde30876352d20e (table default.Table_41f3478fe8c94fbfaf7fcbdee22ce987 [id=e3e121c36ff44ea0a3a98185c8cce381]). Expected version 3 got 0
I20260808 13:35:50.622090 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
W20260808 13:35:50.625692 5771 leader_election.cc:341] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Not found: Tablet not found: daa59a18ae624bc8975bee2c460895a7
I20260808 13:35:50.626163 5771 leader_election.cc:302] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:50.625092 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447]). Expected version 1 got 0
I20260808 13:35:50.627262 8370 raft_consensus.cc:2726] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:50.632033 8632 raft_consensus.cc:482] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.632543 8632 raft_consensus.cc:504] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.631505 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 52deb2e968c04b99a436e04ae78d9969 (table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445]). Expected version 2 got 0
I20260808 13:35:50.637254 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 2d8e4af746774747a70d425068e8dd63 (table default.Table_938ca078ec5e4411b4be33056272983c [id=95f59bd3447c44c291270c448f53d78b]). Expected version 2 got 0
I20260808 13:35:50.640048 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 96bed5e6d1aa49a9b14ba86403c985ce (table default.Table_db41a83ae3f34453af64e73302451c19 [id=1816314c549642e3adc9f39e532ab1ca]). Expected version 1 got 0
I20260808 13:35:50.642990 4832 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f85b4fd30d6c4903946891a399acf7a0 (table default.Table_f042821be4a8421ea3f64f33a071b22d [id=ea2ba343c17d403d8cb0bf9d80bee453]). Expected version 2 got 0
W20260808 13:35:50.652252 5551 leader_election.cc:341] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:50.652716 5551 leader_election.cc:302] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:50.656988 6026 raft_consensus.cc:348] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.657677 6026 raft_consensus.cc:374] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:50.657948 6026 raft_consensus.cc:729] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:50.581686 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d68ab73bae4dd9af46e907b1fd198c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:50.659363 6033 tablet_bootstrap.cc:654] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:50.633616 8524 raft_consensus.cc:686] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 LEADER]: Becoming Leader. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:50.661569 8524 consensus_queue.cc:237] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.629253 8580 raft_consensus.cc:504] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.664525 8580 leader_election.cc:288] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:50.658946 6026 consensus_queue.cc:260] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.675679 6046 tablet_bootstrap.cc:654] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:50.676537 8632 leader_election.cc:288] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:50.679414 5872 raft_consensus.cc:1261] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Refusing update from remote peer 946c9d8462644deba5b45fa724f1923c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260808 13:35:50.679996 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756b5f24ed94a9fbe2a43efc7b8d40d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:50.680322 8370 raft_consensus.cc:482] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.680717 8370 raft_consensus.cc:504] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.682849 8370 leader_election.cc:288] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:50.626652 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_bc6082a17c754033966ecce7c4f9bf26 [id=3a6d52daeb034a88bdcf06112086dff8]
I20260808 13:35:50.694317 5285 leader_election.cc:302] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:50.696938 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]
I20260808 13:35:50.701821 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537]). Expected version 1 got 0
I20260808 13:35:50.702934 8613 raft_consensus.cc:2726] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:50.702965 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 47078us
W20260808 13:35:50.704370 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 52152.8us
I20260808 13:35:50.705569 6026 ts_tablet_manager.cc:1428] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.141s user 0.006s sys 0.000s
I20260808 13:35:50.708635 6026 tablet_bootstrap.cc:492] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:50.709546 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:50.712607 6046 tablet_bootstrap.cc:492] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:50.713053 6046 ts_tablet_manager.cc:1397] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.142s user 0.004s sys 0.008s
I20260808 13:35:50.713454 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b2cce690cd462aa58ceee2bd28c638" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:50.714213 5640 raft_consensus.cc:2379] T d3b2cce690cd462aa58ceee2bd28c638 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:50.715340 8465 raft_consensus.cc:2781] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:50.715880 8465 raft_consensus.cc:686] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:50.716738 8465 consensus_queue.cc:237] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:50.722041 5283 leader_election.cc:341] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:50.723052 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea8f5322a404837b91e1d96749926b7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:50.722492 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 52deb2e968c04b99a436e04ae78d9969 (table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445]). Expected version 2 got 0
W20260808 13:35:50.725859 5550 leader_election.cc:341] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:50.722656 8627 consensus_queue.cc:1035] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:50.730942 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
I20260808 13:35:50.734050 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]). Expected version 2 got 0
I20260808 13:35:50.736137 6026 tablet_bootstrap.cc:654] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:50.741780 4841 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 2d8e4af746774747a70d425068e8dd63 (table default.Table_938ca078ec5e4411b4be33056272983c [id=95f59bd3447c44c291270c448f53d78b]). Expected version 2 got 0
I20260808 13:35:50.761575 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756b5f24ed94a9fbe2a43efc7b8d40d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:50.762282 5630 raft_consensus.cc:2454] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:50.764525 6026 tablet_bootstrap.cc:492] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:50.765116 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 52deb2e968c04b99a436e04ae78d9969 (table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445]). Expected version 2 got 0
I20260808 13:35:50.776623 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:50.782337 6033 tablet_bootstrap.cc:492] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:50.783964 8565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:35:50.784802 8565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:35:50.790007 6026 ts_tablet_manager.cc:1397] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.082s user 0.006s sys 0.004s
W20260808 13:35:50.792459 5551 leader_election.cc:341] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:50.793201 5551 leader_election.cc:302] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:50.793947 8524 raft_consensus.cc:2726] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:50.793816 6026 raft_consensus.cc:348] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.794544 6026 raft_consensus.cc:374] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:50.794821 6026 raft_consensus.cc:729] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:50.795385 6026 consensus_queue.cc:260] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.797472 6026 ts_tablet_manager.cc:1428] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260808 13:35:50.797621 6033 ts_tablet_manager.cc:1397] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.247s user 0.010s sys 0.001s
I20260808 13:35:50.798332 6026 tablet_bootstrap.cc:492] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:50.802160 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b2cce690cd462aa58ceee2bd28c638" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:50.804734 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]
I20260808 13:35:50.808689 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]). Expected version 2 got 0
I20260808 13:35:50.790284 6046 raft_consensus.cc:348] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.812546 6046 raft_consensus.cc:374] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:50.815964 6046 raft_consensus.cc:729] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:50.816182 5285 leader_election.cc:302] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:50.818177 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
W20260808 13:35:50.761970 4841 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: An error occurred while updating sys-tables: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER
I20260808 13:35:50.818581 8627 raft_consensus.cc:2781] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:50.839130 8627 raft_consensus.cc:482] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:50.841496 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea8f5322a404837b91e1d96749926b7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:50.842198 8611 raft_consensus.cc:482] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.842705 8611 raft_consensus.cc:504] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.845958 8611 leader_election.cc:288] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:50.841158 8627 raft_consensus.cc:3037] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:50.851040 4835 catalog_manager.cc:5587] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797) does not have the latest schema for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447]). Expected version 1 got 0
I20260808 13:35:50.818275 6046 consensus_queue.cc:260] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.852207 4835 catalog_manager.cc:5495] T 249dcaf769934b75945d79d827a96be6 P 946c9d8462644deba5b45fa724f1923c reported cstate change: term changed from 0 to 1, leader changed from <none> to 946c9d8462644deba5b45fa724f1923c (127.4.170.1). New cstate: current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } }
I20260808 13:35:50.800098 6033 raft_consensus.cc:348] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.857292 6033 raft_consensus.cc:374] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:50.857656 6033 raft_consensus.cc:729] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:50.859225 6046 ts_tablet_manager.cc:1428] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.146s user 0.006s sys 0.003s
I20260808 13:35:50.859755 8611 raft_consensus.cc:482] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.860190 8611 raft_consensus.cc:504] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.862027 8611 leader_election.cc:288] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:50.863402 5551 leader_election.cc:341] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:50.863788 5551 leader_election.cc:302] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:50.864425 8611 raft_consensus.cc:2726] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:50.866133 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6800eceb96475284fdbffb4d23e5e3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:50.866525 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:50.869997 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a]
I20260808 13:35:50.872342 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea]
I20260808 13:35:50.876451 6046 tablet_bootstrap.cc:492] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:50.877993 6026 tablet_bootstrap.cc:654] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:50.877506 8632 consensus_queue.cc:1035] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:50.899328 8613 raft_consensus.cc:482] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.899791 8613 raft_consensus.cc:504] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.903270 8613 leader_election.cc:288] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:50.913187 8465 raft_consensus.cc:482] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.913822 8465 raft_consensus.cc:504] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.915928 8465 leader_election.cc:288] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:50.916469 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 108651us
I20260808 13:35:50.922484 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6800eceb96475284fdbffb4d23e5e3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:50.916954 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:50.923966 5880 raft_consensus.cc:2454] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 2.
I20260808 13:35:50.928823 8649 tablet.cc:1722] T 8ba098c946794607b2a15bc64098d918 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260808 13:35:50.906620 5285 consensus_peers.cc:487] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a27517560d64a3b8da2cf55980e18a8. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:50.925148 5283 leader_election.cc:302] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:50.936929 5771 leader_election.cc:302] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:50.937186 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:50.941602 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d31516006bc4a65b496dcf363db464a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:50.943638 8465 raft_consensus.cc:2781] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 13:35:50.944080 8465 raft_consensus.cc:482] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:50.944569 8465 raft_consensus.cc:3037] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:35:50.952034 6046 tablet_bootstrap.cc:654] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
E20260808 13:35:50.893317 4835 catalog_manager.cc:5650] Error updating tablets from 946c9d8462644deba5b45fa724f1923c: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER. Tablet report was: is_incremental: true updated_tablets { tablet_id: "a73e98a8b20144508a5a8d451f4bb302" state: RUNNING consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "249dcaf769934b75945d79d827a96be6" state: RUNNING consensus_state { current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { } } updated_tablets { tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" state: RUNNING consensus_state { current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "3a27517560d64a3b8da2cf55980e18a8" state: RUNNING consensus_state { current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "7e8cf06462484c99a860762a82f7ad64" state: RUNNING consensus_state { current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "851a0b374a854efd87dd66e39b9df2d5" state: RUNNING consensus_state { current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26807 live_row_count: 0 } } updated_tablets { tablet_id: "362dbd6990e740b39690537af98aa2f8" state: RUNNING consensus_state { current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "3b1a27f687b54bc592ef4efb72b7247f" state: RUNNING consensus_state { current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26807 live_row_count: 0 } } updated_tablets { tablet_id: "daa59a18ae624bc8975bee2c460895a7" state: RUNNING consensus_state { current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26807 live_row_count: 0 } } updated_tablets { tablet_id: "5663fa6a1bb148c3afc9340477b05a4b" state: RUNNING consensus_state { current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26838 live_row_count: 0 } } updated_tablets { tablet_id: "f85b4fd30d6c4903946891a399acf7a0" state: RUNNING consensus_state { current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 2 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "d0cc688386f3413ea939dda3a75d6541" state: RUNNING consensus_state { current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 1 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "52deb2e968c04b99a436e04ae78d9969" state: RUNNING consensus_state { current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: UNKNOWN } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26729 live_row_count: 0 } } updated_tablets { tablet_id: "adf9a03742a14cfe867c578c466838de" state: RUNNING consensus_state { current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "196d80d806b0432cacf2ca6aa0da8c6b" state: RUNNING consensus_state { current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26838 live_row_count: 0 } } updated_tablets { tablet_id: "72c4ddec62c242c88b3a759d8361cd30" state: RUNNING consensus_state { current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "2bb8c7eeec164485a4e5bb80f9d5fd72" state: RUNNING consensus_state { current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 1 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "f66b38e83dab4bf1a3d2d1e088a96988" state: RUNNING consensus_state { current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "369a942248bd4cec9ee9170fe55601a3" state: STOPPED consensus_state { current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_DELETED } updated_tablets { tablet_id: "4b7284afe0294644a368cf51451c7ab6" state: RUNNING consensus_state { current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 6 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "16a6552c871b4b81b6520cb084d359a2" state: RUNNING consensus_state { current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "275b6aeaad044ea1a16ecd82f63dc59f" state: RUNNING consensus_state { current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "b1847d0fecb843cdbc833ad674a503a4" state: RUNNING consensus_state { current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 1 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "b8ea98afef6841fead641b314ee53a6e" state: RUNNING consensus_state { current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } } } schema_version: 1 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27060 live_row_count: 0 } } updated_tablets { tablet_id: "fcf907a9f19f4d64b27be946011a8ddb" state: RUNNING consensus_state { current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } } } schema_version: 1 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "d9268f5f6dee4112932e106010a3c2be" state: RUNNING consensus_state { current_term: 1 leader_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } } } schema_version: 3 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "c4c083c8e19d4279af600670f5d1d78d" state: RUNNING consensus_state { current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } } } schema_version: 1 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27060 live_row_count: 0 } } updated_tablets { tablet_id: "bcc0c6ab755c4a6887dabeeef9364201" state: RUNNING consensus_state { current_term: 1 leader_uuid: "946c9d8462644deba5b45fa724f1923c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } health_report { overall_health: HEALTHY } } } } schema_version: 4 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27504 live_row_count: 0 } } removed_tablet_ids: "d8b07b7d31ef41e484389c0f007598f0" removed_tablet_ids: "8b57966a9b7b4673ab9f98720cfd456d" removed_tablet_ids: "56d8d5a3ca724141b58e4e2cea23be68" sequence_number: 62
I20260808 13:35:50.945508 8553 raft_consensus.cc:482] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:50.966118 8663 tablet.cc:1722] T 8ba098c946794607b2a15bc64098d918 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260808 13:35:50.967432 8627 raft_consensus.cc:504] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:50.940649 5551 leader_election.cc:341] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:50.972380 5551 leader_election.cc:341] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:50.973206 5551 leader_election.cc:341] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:50.975085 8553 raft_consensus.cc:504] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:50.976262 5872 raft_consensus.cc:3037] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:50.977887 8553 leader_election.cc:288] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:50.881172 6033 consensus_queue.cc:260] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:50.984477 5550 leader_election.cc:341] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:50.984917 5550 leader_election.cc:302] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:50.985311 6026 tablet_bootstrap.cc:492] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:50.985703 6026 ts_tablet_manager.cc:1397] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.188s user 0.003s sys 0.009s
I20260808 13:35:50.986217 8613 raft_consensus.cc:2726] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:50.988061 6026 raft_consensus.cc:348] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.988734 6026 raft_consensus.cc:374] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:50.988986 6026 raft_consensus.cc:729] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:50.989612 6026 consensus_queue.cc:260] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:50.990326 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:50.992144 8652 raft_consensus.cc:3037] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:50.989491 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa86f45770d44e7b993b240607ef2fc" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:50.996873 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532]
W20260808 13:35:50.996852 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 64) took 3472 ms (client timeout 1000 ms). Trace:
I20260808 13:35:51.003253 5877 raft_consensus.cc:3037] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:51.003517 6033 ts_tablet_manager.cc:1428] T a73e98a8b20144508a5a8d451f4bb302 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.206s user 0.005s sys 0.001s
I20260808 13:35:51.004495 6033 tablet_bootstrap.cc:492] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:51.007654 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756b5f24ed94a9fbe2a43efc7b8d40d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:51.008960 5283 leader_election.cc:341] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.009702 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d31516006bc4a65b496dcf363db464a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:51.011392 5550 leader_election.cc:341] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.011827 5550 leader_election.cc:302] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:51.013141 5550 leader_election.cc:341] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.014743 5550 leader_election.cc:302] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:51.016182 8524 raft_consensus.cc:2726] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:51.014242 8612 raft_consensus.cc:2726] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:51.019250 8627 leader_election.cc:288] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:51.019291 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756b5f24ed94a9fbe2a43efc7b8d40d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:51.019917 5630 raft_consensus.cc:3037] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:51.030746 8669 tablet.cc:1722] T 8ba098c946794607b2a15bc64098d918 P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260808 13:35:50.997179 4835 rpcz_store.cc:269] 0808 13:35:47.524063 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.641934 (+117871us) service_pool.cc:227] Handling call
0808 13:35:50.893087 (+3251153us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:50.893110 (+ 23us) write_op.cc:454] Released schema lock
0808 13:35:50.996776 (+103666us) inbound_call.cc:173] Queueing failure response
Related trace 'op':
0808 13:35:50.892857 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:50.892896 (+ 39us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:50.892919 (+ 23us) write_op.cc:454] Released schema lock
0808 13:35:50.892941 (+ 22us) write_op.cc:335] FINISH: Op aborted
Metrics: {"reported_tablets":28,"child_traces":[["op",{}]]}
I20260808 13:35:51.031606 8465 raft_consensus.cc:504] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:51.035490 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 65) took 2419 ms (client timeout 1000 ms). Trace:
W20260808 13:35:51.036072 4835 rpcz_store.cc:269] 0808 13:35:48.615634 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.035301 (+2419667us) service_pool.cc:214] Skipping call since client already timed out
0808 13:35:51.035417 (+ 116us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260808 13:35:51.037266 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 59) took 1699 ms (client timeout 1000 ms). Trace:
W20260808 13:35:51.037669 4835 rpcz_store.cc:269] 0808 13:35:49.337773 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.037096 (+1699323us) service_pool.cc:214] Skipping call since client already timed out
0808 13:35:51.037201 (+ 105us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260808 13:35:51.038056 8627 consensus_queue.cc:1035] T 2fa86f45770d44e7b993b240607ef2fc P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:51.040706 6026 ts_tablet_manager.cc:1428] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.055s user 0.005s sys 0.000s
I20260808 13:35:51.041749 6026 tablet_bootstrap.cc:492] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:51.046205 8652 raft_consensus.cc:2726] T d3b2cce690cd462aa58ceee2bd28c638 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:51.054071 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:51.047034 6033 tablet_bootstrap.cc:654] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:51.055774 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023fe27fc8ce4321a43a1f6037dd9cce" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:51.056335 5866 raft_consensus.cc:3037] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:35:51.026154 4839 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]). Expected version 1 got 0
I20260808 13:35:51.063318 5866 raft_consensus.cc:2454] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 3.
I20260808 13:35:51.064404 5283 leader_election.cc:302] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:51.065366 8627 raft_consensus.cc:2781] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 3 FOLLOWER]: Leader election won for term 3
I20260808 13:35:51.065498 4839 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6]). Expected version 2 got 0
I20260808 13:35:51.066077 8627 raft_consensus.cc:686] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [term 3 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:51.067636 4839 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 538d9a883e9d46668fde30876352d20e (table default.Table_41f3478fe8c94fbfaf7fcbdee22ce987 [id=e3e121c36ff44ea0a3a98185c8cce381]). Expected version 3 got 0
I20260808 13:35:51.067044 8627 consensus_queue.cc:237] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:51.073220 8465 leader_election.cc:288] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 3 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:51.088913 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa86f45770d44e7b993b240607ef2fc" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:51.092262 8465 raft_consensus.cc:482] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.092862 8465 raft_consensus.cc:504] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.095510 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e3245c8ba54f69bc329a5b75115127" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:51.096740 5283 leader_election.cc:341] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:51.038491 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 50) took 1662 ms (client timeout 1000 ms). Trace:
I20260808 13:35:51.084369 8614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:35:51.101640 8614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:51.101922 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e3245c8ba54f69bc329a5b75115127" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:51.102490 5640 raft_consensus.cc:2454] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:51.104002 5285 leader_election.cc:302] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:51.104223 8614 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:35:51.104780 8628 raft_consensus.cc:2781] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:51.105087 8628 raft_consensus.cc:482] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:51.105392 8628 raft_consensus.cc:3037] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:50.998428 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_a446b3e7bb7944b0a5b119dd6a1b6d7c [id=4bfd4a2482364643a6d15f7a638f2eaa]
W20260808 13:35:51.106923 4835 rpcz_store.cc:269] 0808 13:35:49.376585 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.038345 (+1661760us) service_pool.cc:214] Skipping call since client already timed out
0808 13:35:51.038442 (+ 97us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260808 13:35:51.111142 7731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:35:51.115074 8628 raft_consensus.cc:504] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.120947 4839 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447]). Expected version 1 got 0
W20260808 13:35:51.121425 5283 consensus_peers.cc:487] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:51.122117 4839 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 52deb2e968c04b99a436e04ae78d9969 (table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445]). Expected version 2 got 0
I20260808 13:35:51.122131 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e3245c8ba54f69bc329a5b75115127" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:51.123263 5283 leader_election.cc:341] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.120311 8465 leader_election.cc:288] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:51.125312 5772 leader_election.cc:302] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:51.125861 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e3245c8ba54f69bc329a5b75115127" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:51.126960 6026 tablet_bootstrap.cc:654] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:51.128293 5772 consensus_peers.cc:487] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5c0b76c4fdd94a6e9d9bb7709ff1b4be. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:51.126374 5640 raft_consensus.cc:3037] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:51.139314 4839 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 2d8e4af746774747a70d425068e8dd63 (table default.Table_938ca078ec5e4411b4be33056272983c [id=95f59bd3447c44c291270c448f53d78b]). Expected version 2 got 0
I20260808 13:35:51.140192 4839 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 96bed5e6d1aa49a9b14ba86403c985ce (table default.Table_db41a83ae3f34453af64e73302451c19 [id=1816314c549642e3adc9f39e532ab1ca]). Expected version 1 got 0
I20260808 13:35:51.141094 4839 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 37495b29698940edb2924eb612e94bd7 (table default.Table_45fdb943ae1b499e98db51a5ee6589a2 [id=ed0521e8ecae4c609b2a7426e5a565c0]). Expected version 1 got 0
I20260808 13:35:51.141469 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_20c0fcbe614b48089d98badce0ef9eb2 [id=4cbe915d8abf43369d194d4087999bc5]
I20260808 13:35:51.143143 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9]
W20260808 13:35:51.148036 4894 debug-util.cc:398] Leaking SignalData structure 0x7b08000f4420 after lost signal to thread 4792
W20260808 13:35:51.148837 4894 debug-util.cc:398] Leaking SignalData structure 0x7b08000b2720 after lost signal to thread 4899
I20260808 13:35:51.149960 6033 tablet_bootstrap.cc:492] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:51.150349 6033 ts_tablet_manager.cc:1397] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.146s user 0.011s sys 0.001s
I20260808 13:35:51.157689 6033 raft_consensus.cc:348] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:51.159955 6046 tablet_bootstrap.cc:492] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:51.160394 6046 ts_tablet_manager.cc:1397] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.284s user 0.009s sys 0.009s
I20260808 13:35:51.160701 6033 raft_consensus.cc:374] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:51.161975 6033 raft_consensus.cc:729] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:51.162667 6033 consensus_queue.cc:260] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:51.140372 8628 leader_election.cc:288] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:51.174724 8660 raft_consensus.cc:2726] T 2fa86f45770d44e7b993b240607ef2fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:51.177425 6033 ts_tablet_manager.cc:1428] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.027s user 0.006s sys 0.001s
I20260808 13:35:51.181607 6033 tablet_bootstrap.cc:492] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:51.142365 4839 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet d09eb729339e47eb8db88eb6509daab4 (table default.Table_233ac5cc38504a5eb9b9cbf426cc19c7 [id=af41016671a644dcaacf0b2602d83672]). Expected version 1 got 0
I20260808 13:35:51.186187 4833 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60270 (request call id 495) took 7149 ms. Trace:
I20260808 13:35:51.181742 6046 raft_consensus.cc:348] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.186547 4833 rpcz_store.cc:276] 0808 13:35:44.037202 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.980585 (+1943383us) service_pool.cc:227] Handling call
0808 13:35:46.059234 (+ 78649us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:46.059900 (+ 666us) catalog_manager.cc:3616] Apply alter schema
0808 13:35:46.082440 (+ 22540us) catalog_manager.cc:3639] Acquired catalog manager lock
0808 13:35:46.082690 (+ 250us) catalog_manager.cc:3786] Updating metadata on disk
0808 13:35:47.287496 (+1204806us) catalog_manager.cc:3817] Committing alterations to in-memory state
0808 13:35:51.186120 (+3898624us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:47.155713 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:47.156769 (+ 1056us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:47.156830 (+ 61us) write_op.cc:435] Acquired schema lock
0808 13:35:47.156874 (+ 44us) tablet.cc:660] Decoding operations
0808 13:35:47.157582 (+ 708us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:47.157720 (+ 138us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:47.157778 (+ 58us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:47.219352 (+ 61574us) tablet.cc:699] Row locks acquired
0808 13:35:47.219368 (+ 16us) write_op.cc:260] PREPARE: finished
0808 13:35:47.219636 (+ 268us) write_op.cc:270] Start()
0808 13:35:47.219824 (+ 188us) write_op.cc:276] Timestamp: P: 1786196147219507 usec, L: 0
0808 13:35:47.219869 (+ 45us) op_driver.cc:331] REPLICATION: starting
0808 13:35:47.236651 (+ 16782us) log.cc:842] Serialized 405 byte log entry
0808 13:35:47.283965 (+ 47314us) op_driver.cc:444] REPLICATION: finished
0808 13:35:47.285178 (+ 1213us) write_op.cc:301] APPLY: starting
0808 13:35:47.285266 (+ 88us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:47.285487 (+ 221us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:47.285684 (+ 197us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:47.286225 (+ 541us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:47.286564 (+ 339us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:47.286681 (+ 117us) write_op.cc:312] APPLY: finished
0808 13:35:47.286939 (+ 258us) log.cc:842] Serialized 23 byte log entry
0808 13:35:47.287119 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.287219 (+ 100us) write_op.cc:454] Released schema lock
0808 13:35:47.287265 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:47.287943 (+ 678us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.288007 (+ 64us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":218,"child_traces":[["op",{"apply.queue_time_us":930,"apply.run_cpu_time_us":2392,"apply.run_wall_time_us":3050,"mutex_wait_us":416279,"num_ops":1,"prepare.queue_time_us":1071526,"prepare.run_cpu_time_us":4667,"prepare.run_wall_time_us":81414,"raft.queue_time_us":21331,"raft.run_cpu_time_us":1587,"raft.run_wall_time_us":51111,"replication_time_us":63988}]]}
I20260808 13:35:51.187037 5640 raft_consensus.cc:2454] T b8e3245c8ba54f69bc329a5b75115127 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:51.189162 6046 raft_consensus.cc:374] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:51.194479 4776 master-stress-test.cc:580] Killed test thread 5/10
I20260808 13:35:51.194793 4776 master-stress-test.cc:578] Killing test thread 6/10
I20260808 13:35:51.195045 4776 master-stress-test.cc:580] Killed test thread 6/10
I20260808 13:35:51.195243 4776 master-stress-test.cc:578] Killing test thread 7/10
I20260808 13:35:51.195832 4776 master-stress-test.cc:580] Killed test thread 7/10
I20260808 13:35:51.196045 4776 master-stress-test.cc:578] Killing test thread 8/10
I20260808 13:35:51.196384 5285 leader_election.cc:302] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:51.197393 8628 raft_consensus.cc:2781] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:51.197827 8628 raft_consensus.cc:686] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:51.198638 8628 consensus_queue.cc:237] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.202349 6033 tablet_bootstrap.cc:654] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:51.194881 6046 raft_consensus.cc:729] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:35:51.208716 4841 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: An error occurred while updating sys-tables: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER
I20260808 13:35:51.205639 6046 consensus_queue.cc:260] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.216315 5831 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c2b34c95244472beec43643e60ff5f with delete_type TABLET_DATA_DELETED (replaced by tablet b314734698ed423ba9cf4134db5846ec) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:51.216599 5630 raft_consensus.cc:2454] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:51.217300 5826 tablet_service.cc:1514] Processing DeleteTablet for tablet 851a0b374a854efd87dd66e39b9df2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:51.217826 5285 leader_election.cc:302] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:51.218051 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 9dee4b3b14264f3aad0896f629ab5b27 (table default.Table_4cb4f6347fda4d268fd87065c9b063ea [id=11f8a63441444ac1a7aa7165cbe4b696], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9dee4b3b14264f3aad0896f629ab5b27 consensus state has no leader
I20260808 13:35:51.218623 8465 raft_consensus.cc:2781] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:51.219260 8465 raft_consensus.cc:686] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:51.220220 8465 consensus_queue.cc:237] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.224815 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_3961834935a84f0a9646f77db54765ee [id=5115ecc58364490c953100f78c49facf]
I20260808 13:35:51.227378 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_b9063cf92547431f9c63a652c79fbaae [id=53aad8e2f83c4f8d87add3777f909fa5]
W20260808 13:35:51.229944 4837 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: An error occurred while updating sys-tables: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER
I20260808 13:35:51.238919 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 369a942248bd4cec9ee9170fe55601a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:51.239954 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 362dbd6990e740b39690537af98aa2f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:51.235405 5826 tablet_service.cc:1514] Processing DeleteTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:22 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:51.240682 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8cf06462484c99a860762a82f7ad64 with delete_type TABLET_DATA_DELETED (replaced by tablet 16736500f94c43d3a3163f61b7606aad) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:51.238155 6046 ts_tablet_manager.cc:1428] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.077s user 0.008s sys 0.003s
I20260808 13:35:51.239833 8628 raft_consensus.cc:482] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:51.245402 4837 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: An error occurred while updating sys-tables: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER
I20260808 13:35:51.246881 6033 tablet_bootstrap.cc:492] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:51.247262 6033 ts_tablet_manager.cc:1397] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.066s user 0.010s sys 0.001s
I20260808 13:35:51.248435 6046 tablet_bootstrap.cc:492] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:51.249585 6033 raft_consensus.cc:348] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.250659 6033 raft_consensus.cc:374] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:51.243767 8628 raft_consensus.cc:504] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.253157 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a27517560d64a3b8da2cf55980e18a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:51.250939 6033 raft_consensus.cc:729] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:51.262115 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba5fa465ec04bbb801a821a71a8f87d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:51.263554 5283 leader_election.cc:341] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.266671 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba5fa465ec04bbb801a821a71a8f87d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:51.266265 8628 leader_election.cc:288] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:51.269951 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet daa59a18ae624bc8975bee2c460895a7 with delete_type TABLET_DATA_DELETED (replaced by tablet cb7f6487e28f4aaa94d448be71f8a3c2) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:51.260615 6033 consensus_queue.cc:260] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.271553 8628 raft_consensus.cc:482] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.272500 6033 ts_tablet_manager.cc:1428] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
W20260808 13:35:51.237601 4842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 47) took 6783 ms (client timeout 1000 ms). Trace:
I20260808 13:35:51.273152 5630 raft_consensus.cc:2454] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
W20260808 13:35:51.272890 4842 rpcz_store.cc:269] 0808 13:35:44.454120 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.125839 (+671719us) service_pool.cc:227] Handling call
0808 13:35:51.237530 (+6111691us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":24456,"reported_tablets":31}
I20260808 13:35:51.279430 5285 leader_election.cc:302] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:51.281055 8627 raft_consensus.cc:2781] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:51.281383 8627 raft_consensus.cc:482] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:51.281680 8627 raft_consensus.cc:3037] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:51.281679 6033 tablet_bootstrap.cc:492] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:51.282261 4840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 48) took 5477 ms (client timeout 1000 ms). Trace:
I20260808 13:35:51.287762 6033 tablet_bootstrap.cc:654] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:51.290087 6026 tablet_bootstrap.cc:492] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:51.297598 8627 raft_consensus.cc:504] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.275098 8628 raft_consensus.cc:504] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.311352 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba5fa465ec04bbb801a821a71a8f87d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
W20260808 13:35:51.312811 5283 leader_election.cc:341] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.313697 6033 tablet_bootstrap.cc:492] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:51.314138 6033 ts_tablet_manager.cc:1397] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.004s
I20260808 13:35:51.316674 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_6fc58ab15c1b44c69500f3ad87062bcd [id=562d628bce6b4d96b2013e847c87a8cb]
I20260808 13:35:51.316638 6033 raft_consensus.cc:348] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.318691 6033 raft_consensus.cc:374] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:51.319039 6033 raft_consensus.cc:729] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:51.319136 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_cd53ce0f4bbd47f5a510f440e7e066c8 [id=59ed02534c50457ab73d5c2757bda959]
I20260808 13:35:51.320530 6026 ts_tablet_manager.cc:1397] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.279s user 0.007s sys 0.003s
I20260808 13:35:51.321028 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_0e051b613c6e452f89c46cb5aff7f7dd [id=5b28fc73db9a4600bfa8c43e9929b4cb]
I20260808 13:35:51.325656 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa049ef916b44458be43ec4e2ccaf1c" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:51.326012 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa049ef916b44458be43ec4e2ccaf1c" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:51.326591 5640 raft_consensus.cc:2454] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
W20260808 13:35:51.326936 5283 leader_election.cc:341] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.327638 5285 leader_election.cc:302] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:51.328378 8465 raft_consensus.cc:2781] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:51.328701 8465 raft_consensus.cc:482] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:51.329015 8465 raft_consensus.cc:3037] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:51.320271 6033 consensus_queue.cc:260] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.329488 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba5fa465ec04bbb801a821a71a8f87d" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:51.330242 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: Sending DeleteTablet for 3 replicas of tablet 4ff163c36c6d42238cce2396102f9a68
I20260808 13:35:51.331780 4838 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60270 (request call id 497) took 6866 ms. Trace:
I20260808 13:35:51.332377 6033 ts_tablet_manager.cc:1428] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260808 13:35:51.333256 6033 tablet_bootstrap.cc:492] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:51.332083 4838 rpcz_store.cc:276] 0808 13:35:44.465119 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:46.239440 (+1774321us) service_pool.cc:227] Handling call
0808 13:35:46.508738 (+269298us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0808 13:35:46.564304 (+ 55566us) catalog_manager.cc:2752] Modifying in-memory table state
0808 13:35:46.564503 (+ 199us) catalog_manager.cc:2761] Locking tablets
0808 13:35:46.807803 (+243300us) catalog_manager.cc:2775] Removing table and tablets from system table
0808 13:35:47.453456 (+645653us) catalog_manager.cc:2793] Removing table from by-name map
0808 13:35:47.538275 (+ 84819us) catalog_manager.cc:2809] Committing in-memory state
0808 13:35:51.330140 (+3791865us) catalog_manager.cc:2813] Aborting table tasks
0808 13:35:51.331717 (+ 1577us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:47.237331 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:47.238299 (+ 968us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:47.238359 (+ 60us) write_op.cc:435] Acquired schema lock
0808 13:35:47.238457 (+ 98us) tablet.cc:660] Decoding operations
0808 13:35:47.239263 (+ 806us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:47.239368 (+ 105us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:47.239428 (+ 60us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:47.311474 (+ 72046us) tablet.cc:699] Row locks acquired
0808 13:35:47.311525 (+ 51us) write_op.cc:260] PREPARE: finished
0808 13:35:47.311783 (+ 258us) write_op.cc:270] Start()
0808 13:35:47.311973 (+ 190us) write_op.cc:276] Timestamp: P: 1786196147311634 usec, L: 0
0808 13:35:47.312023 (+ 50us) op_driver.cc:331] REPLICATION: starting
0808 13:35:47.313180 (+ 1157us) log.cc:842] Serialized 725 byte log entry
0808 13:35:47.449217 (+136037us) op_driver.cc:444] REPLICATION: finished
0808 13:35:47.451074 (+ 1857us) write_op.cc:301] APPLY: starting
0808 13:35:47.451177 (+ 103us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:47.451412 (+ 235us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:47.451612 (+ 200us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:47.452039 (+ 427us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:47.452284 (+ 245us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:47.452369 (+ 85us) write_op.cc:312] APPLY: finished
0808 13:35:47.452721 (+ 352us) log.cc:842] Serialized 29 byte log entry
0808 13:35:47.452952 (+ 231us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.453075 (+ 123us) write_op.cc:454] Released schema lock
0808 13:35:47.453135 (+ 60us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:47.454074 (+ 939us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.454153 (+ 79us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1584,"apply.run_cpu_time_us":2879,"apply.run_wall_time_us":3455,"num_ops":2,"prepare.queue_time_us":390486,"prepare.run_cpu_time_us":5200,"prepare.run_wall_time_us":260061,"raft.queue_time_us":123583,"raft.run_cpu_time_us":1175,"raft.run_wall_time_us":37926,"replication_time_us":137067}]]}
I20260808 13:35:51.336140 8465 raft_consensus.cc:504] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.337067 4838 master_service.cc:946] ReplaceTablet: received request to replace tablet 433fcbc6cdca458b8404ae5862f280a3 from {username='slave'} at 127.0.0.1:60270
I20260808 13:35:51.338253 4838 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60270 (request call id 502) took 2690 ms. Trace:
I20260808 13:35:51.339327 4838 rpcz_store.cc:276] 0808 13:35:48.647825 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.336959 (+2689134us) service_pool.cc:227] Handling call
0808 13:35:51.338191 (+ 1232us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:51.339552 4776 master-stress-test.cc:580] Killed test thread 8/10
I20260808 13:35:51.339811 4776 master-stress-test.cc:578] Killing test thread 9/10
I20260808 13:35:51.340863 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa049ef916b44458be43ec4e2ccaf1c" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:51.341703 5630 raft_consensus.cc:3037] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:51.347383 5962 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.4.170.62:34069)
I20260808 13:35:51.347541 5335 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff163c36c6d42238cce2396102f9a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:46 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:51.352701 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff163c36c6d42238cce2396102f9a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:46 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:51.354879 6046 tablet_bootstrap.cc:654] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:51.355296 5283 leader_election.cc:341] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.338485 6033 tablet_bootstrap.cc:654] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:51.338613 4832 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: An error occurred while updating sys-tables: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER
W20260808 13:35:51.367637 4836 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: An error occurred while updating sys-tables: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER
I20260808 13:35:51.365453 8627 leader_election.cc:288] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:51.367926 8628 leader_election.cc:288] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:51.375046 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff163c36c6d42238cce2396102f9a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:46 UTC) from {username='slave'} at 127.0.0.1:54830
I20260808 13:35:51.376958 8700 tablet_replica.cc:330] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:51.379781 8700 raft_consensus.cc:2229] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 13:35:51.386219 4832 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: An error occurred while updating sys-tables: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER
I20260808 13:35:51.389740 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa049ef916b44458be43ec4e2ccaf1c" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:51.389336 8465 leader_election.cc:288] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:51.390338 5634 raft_consensus.cc:3037] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:51.397698 5634 raft_consensus.cc:2454] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:51.398742 5285 leader_election.cc:302] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:51.400131 8465 raft_consensus.cc:2781] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:51.400626 8465 raft_consensus.cc:686] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:51.401508 8465 consensus_queue.cc:237] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.403970 8700 raft_consensus.cc:2258] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:51.406522 8632 consensus_queue.cc:1035] T a04d3dd028da4e7f86689e819548d0c5 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:51.410938 8663 tablet.cc:1722] T 275b6aeaad044ea1a16ecd82f63dc59f P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:51.416447 8649 tablet.cc:1722] T 275b6aeaad044ea1a16ecd82f63dc59f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:51.424014 8613 raft_consensus.cc:482] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.424520 8613 raft_consensus.cc:504] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.426426 8613 leader_election.cc:288] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:51.427709 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7e8060cab44121acf6dadd2c2873b3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:51.429407 5550 leader_election.cc:341] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.431113 6033 tablet_bootstrap.cc:492] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:51.431589 6033 ts_tablet_manager.cc:1397] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.098s user 0.021s sys 0.000s
I20260808 13:35:51.322563 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_5128302bae8343618fb6509466b51c71 [id=5c3825e556e04b17b0437380aa662748]
I20260808 13:35:51.434010 6033 raft_consensus.cc:348] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.447651 6033 raft_consensus.cc:374] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:51.447953 6033 raft_consensus.cc:729] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:51.448637 6033 consensus_queue.cc:260] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.453161 6033 ts_tablet_manager.cc:1428] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260808 13:35:51.453513 8698 tablet_replica.cc:330] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:51.454775 8698 raft_consensus.cc:2229] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:51.456857 8698 raft_consensus.cc:2258] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:51.447183 8700 ts_tablet_manager.cc:1905] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:51.458827 8669 tablet.cc:1722] T 275b6aeaad044ea1a16ecd82f63dc59f P 946c9d8462644deba5b45fa724f1923c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:35:51.459872 6026 raft_consensus.cc:348] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.460917 6026 raft_consensus.cc:374] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:51.463164 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7e8060cab44121acf6dadd2c2873b3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:51.467834 8652 raft_consensus.cc:482] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.468291 8652 raft_consensus.cc:504] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.469354 5335 tablet_service.cc:1514] Processing DeleteTablet for tablet 369a942248bd4cec9ee9170fe55601a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:21 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:51.470258 5335 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8cf06462484c99a860762a82f7ad64 with delete_type TABLET_DATA_DELETED (replaced by tablet 16736500f94c43d3a3163f61b7606aad) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:51.472430 8654 consensus_queue.cc:1035] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:51.286383 4840 rpcz_store.cc:269] 0808 13:35:45.805203 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.838124 (+ 32921us) service_pool.cc:227] Handling call
0808 13:35:51.282172 (+5444048us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:47.795333 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:47.795758 (+ 425us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:47.795802 (+ 44us) write_op.cc:435] Acquired schema lock
0808 13:35:47.795814 (+ 12us) tablet.cc:660] Decoding operations
0808 13:35:47.796145 (+ 331us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:47.796194 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:47.796217 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:35:47.796397 (+ 180us) tablet.cc:699] Row locks acquired
0808 13:35:47.796410 (+ 13us) write_op.cc:260] PREPARE: finished
0808 13:35:47.796534 (+ 124us) write_op.cc:270] Start()
0808 13:35:47.796626 (+ 92us) write_op.cc:276] Timestamp: P: 1786196147796453 usec, L: 0
0808 13:35:47.796642 (+ 16us) op_driver.cc:331] REPLICATION: starting
0808 13:35:47.803881 (+ 7239us) log.cc:842] Serialized 527 byte log entry
0808 13:35:47.837625 (+ 33744us) op_driver.cc:444] REPLICATION: finished
0808 13:35:47.838076 (+ 451us) write_op.cc:301] APPLY: starting
0808 13:35:47.838220 (+ 144us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:47.838450 (+ 230us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:47.838719 (+ 269us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:47.839182 (+ 463us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:47.839452 (+ 270us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:35:47.839571 (+ 119us) write_op.cc:312] APPLY: finished
0808 13:35:47.839915 (+ 344us) log.cc:842] Serialized 23 byte log entry
0808 13:35:47.840174 (+ 259us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.840282 (+ 108us) write_op.cc:454] Released schema lock
0808 13:35:47.840340 (+ 58us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:47.840613 (+ 273us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:47.840657 (+ 44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":87254,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":178,"apply.run_cpu_time_us":2824,"apply.run_wall_time_us":2823,"num_ops":1,"prepare.queue_time_us":241184,"prepare.run_cpu_time_us":2677,"prepare.run_wall_time_us":86908,"raft.queue_time_us":330,"raft.run_cpu_time_us":1331,"raft.run_wall_time_us":1331,"replication_time_us":40845}]]}
I20260808 13:35:51.472772 8652 leader_election.cc:288] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:51.439282 4841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 49) took 3858 ms (client timeout 1000 ms). Trace:
W20260808 13:35:51.475186 4841 rpcz_store.cc:269] 0808 13:35:47.581270 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.740844 (+159574us) service_pool.cc:227] Handling call
0808 13:35:50.761736 (+3020892us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:50.761759 (+ 23us) write_op.cc:454] Released schema lock
0808 13:35:51.208518 (+446759us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.208538 (+ 20us) write_op.cc:454] Released schema lock
0808 13:35:51.439198 (+230660us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:50.761524 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:50.761572 (+ 48us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:50.761590 (+ 18us) write_op.cc:454] Released schema lock
0808 13:35:50.761610 (+ 20us) write_op.cc:335] FINISH: Op aborted
Related trace 'op':
0808 13:35:51.208333 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:51.208365 (+ 32us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.208386 (+ 21us) write_op.cc:454] Released schema lock
0808 13:35:51.208403 (+ 17us) write_op.cc:335] FINISH: Op aborted
Metrics: {"mutex_wait_us":371,"reported_tablets":39,"child_traces":[["op",{}],["op",{}]]}
I20260808 13:35:51.461966 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_da97795db35e47d8aed30d142ba853d2 [id=5d87520fa77a4254878c3f2d49f6592b]
I20260808 13:35:51.481144 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa049ef916b44458be43ec4e2ccaf1c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:51.484730 5644 raft_consensus.cc:2379] T eaa049ef916b44458be43ec4e2ccaf1c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in current term 1: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:51.485332 6026 raft_consensus.cc:729] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:51.486275 6026 consensus_queue.cc:260] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.488008 8524 raft_consensus.cc:482] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.488404 8524 raft_consensus.cc:504] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.488823 6026 ts_tablet_manager.cc:1428] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.168s user 0.005s sys 0.001s
I20260808 13:35:51.489693 6026 tablet_bootstrap.cc:492] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:51.490235 8524 leader_election.cc:288] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:51.491752 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37495b29698940edb2924eb612e94bd7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:51.492919 5550 leader_election.cc:341] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:51.480479 4836 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: An error occurred while updating sys-tables: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER
I20260808 13:35:51.498596 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a27517560d64a3b8da2cf55980e18a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:50600
W20260808 13:35:51.472510 4835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:51951 (request call id 51) took 1150 ms (client timeout 1000 ms). Trace:
I20260808 13:35:51.508913 8684 tablet_replica.cc:330] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
W20260808 13:35:51.508679 4835 rpcz_store.cc:269] 0808 13:35:50.322423 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.108630 (+786207us) service_pool.cc:227] Handling call
0808 13:35:51.472426 (+363796us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:51.510035 6033 tablet_bootstrap.cc:492] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:51.515071 8524 raft_consensus.cc:482] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.515476 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_d56fc9f7ea3a43f98fe5a3057b51dbac [id=60103bc8b4444b688fdf2baab858ebf5]
I20260808 13:35:51.515547 8524 raft_consensus.cc:504] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.517470 8524 leader_election.cc:288] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:51.519219 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3272708a835c432c878cc330e44490c5" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:51.520483 5550 leader_election.cc:341] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.521710 6033 tablet_bootstrap.cc:654] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:51.522583 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa049ef916b44458be43ec4e2ccaf1c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:51.525449 4837 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:40485 (request call id 63) took 5640 ms (client timeout 1000 ms). Trace:
I20260808 13:35:51.527819 5630 raft_consensus.cc:2454] T 1ba5fa465ec04bbb801a821a71a8f87d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:51.527840 8698 ts_tablet_manager.cc:1905] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:51.528980 5285 leader_election.cc:302] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:51.529708 8628 raft_consensus.cc:2781] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:51.533169 5771 leader_election.cc:302] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:51.534251 8660 raft_consensus.cc:3037] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:51.540151 5551 leader_election.cc:341] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.540726 5551 leader_election.cc:302] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:51.541649 8660 raft_consensus.cc:2726] T eaa049ef916b44458be43ec4e2ccaf1c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:51.551829 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_854eb2acc74c43a688e89cd2fb1f3b96 [id=65c22acaec6e40f78aa828752330ed2a]
I20260808 13:35:51.554806 8524 raft_consensus.cc:2726] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:51.556846 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 851a0b374a854efd87dd66e39b9df2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:51.557598 5336 tablet_service.cc:1514] Processing DeleteTablet for tablet 362dbd6990e740b39690537af98aa2f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:51.559056 8524 raft_consensus.cc:482] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.559266 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 851a0b374a854efd87dd66e39b9df2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:19 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:51.559590 8524 raft_consensus.cc:504] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.562700 8524 leader_election.cc:288] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:51.563378 5872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20113491b108476b95b8f32b69176d0e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:51.564695 5550 leader_election.cc:341] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.565378 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet daa59a18ae624bc8975bee2c460895a7 with delete_type TABLET_DATA_DELETED (replaced by tablet cb7f6487e28f4aaa94d448be71f8a3c2) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:51.575209 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 362dbd6990e740b39690537af98aa2f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:24 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:51.579368 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c2b34c95244472beec43643e60ff5f with delete_type TABLET_DATA_DELETED (replaced by tablet b314734698ed423ba9cf4134db5846ec) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:51.588541 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_6932f06e7e1e4ec79c76ad6efea73902 [id=665fac81069c47c2b2b787ed500d3a14]
I20260808 13:35:51.590097 8684 raft_consensus.cc:2229] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:51.590262 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_e9d344caa3c2448784d6108b083eca10 [id=6697c24737964dc09cde7d28b2e12447]
I20260808 13:35:51.591262 8684 raft_consensus.cc:2258] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:51.591701 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_d5bb15ded3d442699c050da99887a456 [id=67953edd9d3c47b8b489f036a64c74ae]
I20260808 13:35:51.594033 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_7e9aac444b5a43f29dc7608e6ef379f3 [id=6a2ea5033865443d85ac6e6429054471]
I20260808 13:35:51.594276 8684 ts_tablet_manager.cc:1905] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:51.530292 8628 raft_consensus.cc:686] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:51.598865 5356 tablet_service.cc:1514] Processing DeleteTablet for tablet daa59a18ae624bc8975bee2c460895a7 with delete_type TABLET_DATA_DELETED (replaced by tablet cb7f6487e28f4aaa94d448be71f8a3c2) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:51.598881 5351 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a27517560d64a3b8da2cf55980e18a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:18 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:51.599613 8628 consensus_queue.cc:237] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.622414 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:31 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:51.625339 5872 raft_consensus.cc:1226] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 3 FOLLOWER]: Rejecting Update request from peer 5cd987c3dbdb4168beb73cd7e577c7cb for earlier term 2. Current term is 3. Ops: []
I20260808 13:35:51.626997 8524 consensus_queue.cc:1046] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260808 13:35:51.627913 8524 raft_consensus.cc:3032] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 LEADER]: Stepping down as leader of term 2
I20260808 13:35:51.628213 8524 raft_consensus.cc:729] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Running, Role: LEADER
I20260808 13:35:51.629294 6033 tablet_bootstrap.cc:492] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:51.628957 8524 consensus_queue.cc:260] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:51.629778 6033 ts_tablet_manager.cc:1397] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.120s user 0.014s sys 0.000s
I20260808 13:35:51.630137 8524 raft_consensus.cc:3037] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:35:51.633551 6033 raft_consensus.cc:348] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.634464 6033 raft_consensus.cc:374] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:51.634846 6033 raft_consensus.cc:729] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:51.635555 6033 consensus_queue.cc:260] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.634806 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 0756b5f24ed94a9fbe2a43efc7b8d40d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:28 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:51.637656 6033 ts_tablet_manager.cc:1428] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260808 13:35:51.637975 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet ba39706cff65404485959865fff1772a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:51.638949 6033 tablet_bootstrap.cc:492] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:51.647434 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3272708a835c432c878cc330e44490c5" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:51.668538 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_0ae56405f82f446dbd5ed65b0062fc62 [id=6a9d5e1c30ff4f5490feb1eb08805fee]
I20260808 13:35:51.674469 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20113491b108476b95b8f32b69176d0e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:51.654243 5335 tablet_service.cc:1514] Processing DeleteTablet for tablet 196d80d806b0432cacf2ca6aa0da8c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:22 UTC) from {username='slave'} at 127.0.0.1:46328
I20260808 13:35:51.679318 6026 tablet_bootstrap.cc:654] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:51.681272 8660 raft_consensus.cc:482] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.681818 8660 raft_consensus.cc:504] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.684500 8660 leader_election.cc:288] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:51.685868 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37495b29698940edb2924eb612e94bd7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:51.694178 8627 consensus_queue.cc:1035] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:51.699045 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d31516006bc4a65b496dcf363db464a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:51.700506 5630 raft_consensus.cc:2454] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:51.701333 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet ba39706cff65404485959865fff1772a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:51.701921 5772 leader_election.cc:302] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:51.702477 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet de6800eceb96475284fdbffb4d23e5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:41 UTC) from {username='slave'} at 127.0.0.1:50600
I20260808 13:35:51.703450 8652 raft_consensus.cc:2781] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:51.703843 8652 raft_consensus.cc:482] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:51.704211 8652 raft_consensus.cc:3037] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:51.638266 4836 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 56) took 6287 ms (client timeout 1000 ms). Trace:
I20260808 13:35:51.703543 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_1ab5d21bca4a47978a8f5483f2ba10b2 [id=6d0c2c16340f4cd8805c7692e6275a66]
W20260808 13:35:51.707087 4836 rpcz_store.cc:269] 0808 13:35:45.351263 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.797560 (+446297us) service_pool.cc:227] Handling call
0808 13:35:51.367429 (+5569869us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.367449 (+ 20us) write_op.cc:454] Released schema lock
0808 13:35:51.480265 (+112816us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.480287 (+ 22us) write_op.cc:454] Released schema lock
0808 13:35:51.638177 (+157890us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:51.367232 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:51.367278 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.367297 (+ 19us) write_op.cc:454] Released schema lock
0808 13:35:51.367315 (+ 18us) write_op.cc:335] FINISH: Op aborted
Related trace 'op':
0808 13:35:51.479962 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:51.479996 (+ 34us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.480018 (+ 22us) write_op.cc:454] Released schema lock
0808 13:35:51.480035 (+ 17us) write_op.cc:335] FINISH: Op aborted
Metrics: {"mutex_wait_us":29146,"reported_tablets":36,"child_traces":[["op",{}],["op",{}]]}
I20260808 13:35:51.710443 8627 consensus_queue.cc:1035] T d3b2cce690cd462aa58ceee2bd28c638 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:35:51.525779 4837 rpcz_store.cc:269] 0808 13:35:45.885009 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:45.944842 (+ 59833us) service_pool.cc:227] Handling call
0808 13:35:51.229710 (+5284868us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.229735 (+ 25us) write_op.cc:454] Released schema lock
0808 13:35:51.245172 (+ 15437us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.245199 (+ 27us) write_op.cc:454] Released schema lock
0808 13:35:51.525377 (+280178us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:47.946579 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:35:47.955526 (+ 8947us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:35:47.955574 (+ 48us) write_op.cc:435] Acquired schema lock
0808 13:35:47.955618 (+ 44us) tablet.cc:660] Decoding operations
0808 13:35:47.956355 (+ 737us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:35:47.956450 (+ 95us) write_op.cc:641] Partition lock acquired for write op
0808 13:35:47.956508 (+ 58us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:35:47.957085 (+ 577us) tablet.cc:699] Row locks acquired
0808 13:35:47.957130 (+ 45us) write_op.cc:260] PREPARE: finished
0808 13:35:47.957337 (+ 207us) write_op.cc:270] Start()
0808 13:35:47.957460 (+ 123us) write_op.cc:276] Timestamp: P: 1786196147957218 usec, L: 0
0808 13:35:47.957509 (+ 49us) op_driver.cc:331] REPLICATION: starting
0808 13:35:47.958595 (+ 1086us) log.cc:842] Serialized 901 byte log entry
0808 13:35:48.022360 (+ 63765us) op_driver.cc:444] REPLICATION: finished
0808 13:35:48.022653 (+ 293us) write_op.cc:301] APPLY: starting
0808 13:35:48.022714 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0808 13:35:48.022852 (+ 138us) tablet.cc:1368] finished BulkCheckPresence
0808 13:35:48.022978 (+ 126us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:35:48.023321 (+ 343us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:35:48.023463 (+ 142us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0808 13:35:48.023512 (+ 49us) write_op.cc:312] APPLY: finished
0808 13:35:48.023746 (+ 234us) log.cc:842] Serialized 29 byte log entry
0808 13:35:48.023908 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:48.023977 (+ 69us) write_op.cc:454] Released schema lock
0808 13:35:48.024011 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0808 13:35:48.024184 (+ 173us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:48.024205 (+ 21us) write_op.cc:454] Released schema lock
Related trace 'op':
0808 13:35:51.229500 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:51.229541 (+ 41us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.229565 (+ 24us) write_op.cc:454] Released schema lock
0808 13:35:51.229588 (+ 23us) write_op.cc:335] FINISH: Op aborted
Related trace 'op':
0808 13:35:51.244985 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:51.245019 (+ 34us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.245038 (+ 19us) write_op.cc:454] Released schema lock
0808 13:35:51.245057 (+ 19us) write_op.cc:335] FINISH: Op aborted
Metrics: {"mutex_wait_us":371847,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":109,"apply.run_cpu_time_us":1750,"apply.run_wall_time_us":1754,"num_ops":2,"prepare.queue_time_us":11663,"prepare.run_cpu_time_us":4684,"prepare.run_wall_time_us":101106,"raft.queue_time_us":164,"raft.run_cpu_time_us":1270,"raft.run_wall_time_us":1270,"replication_time_us":64768}],["op",{}],["op",{}]]}
W20260808 13:35:51.614598 4839 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: An error occurred while updating sys-tables: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER
W20260808 13:35:51.703788 4832 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 57) took 4808 ms (client timeout 1000 ms). Trace:
W20260808 13:35:51.727108 4832 rpcz_store.cc:269] 0808 13:35:46.895809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:47.130816 (+235007us) service_pool.cc:227] Handling call
0808 13:35:47.355654 (+224838us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b5400021554. stack: 00007f22752a8033 00007f22752a7f30 00007f2274e29534 00000000004db2b4 00000000004db249 00007f227c1b4af0 00007f227c237f2a 00007f227e103d6c 00007f227e1fdc3e 00007f227bd34fb4 00007f227bd34f42 00007f227bd34e71 00007f227bd34dec 00007f227bd340a2 00007f2279fde94c 00007f2279fddd86
0808 13:35:51.338391 (+3982737us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.338413 (+ 22us) write_op.cc:454] Released schema lock
0808 13:35:51.385980 (+ 47567us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.386009 (+ 29us) write_op.cc:454] Released schema lock
0808 13:35:51.703703 (+317694us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:51.338097 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:51.338135 (+ 38us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.338161 (+ 26us) write_op.cc:454] Released schema lock
0808 13:35:51.338185 (+ 24us) write_op.cc:335] FINISH: Op aborted
Related trace 'op':
0808 13:35:51.385743 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:51.385787 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.385817 (+ 30us) write_op.cc:454] Released schema lock
0808 13:35:51.385842 (+ 25us) write_op.cc:335] FINISH: Op aborted
Metrics: {"reported_tablets":39,"spinlock_wait_cycles":73532800,"child_traces":[["op",{}],["op",{}]]}
I20260808 13:35:51.731380 8652 raft_consensus.cc:504] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.754067 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_577d84019e77413d9ff7373a8315d384 [id=6f200cd5eaff4a63a8b2672962db8c78]
W20260808 13:35:51.754292 4839 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c: An error occurred while updating sys-tables: Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER
I20260808 13:35:51.698549 8698 ts_tablet_manager.cc:1918] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:51.760337 8698 log.cc:1196] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/4ff163c36c6d42238cce2396102f9a68
I20260808 13:35:51.761191 6046 tablet_bootstrap.cc:492] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:51.763309 6046 ts_tablet_manager.cc:1397] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.517s user 0.010s sys 0.028s
I20260808 13:35:51.764765 8698 ts_tablet_manager.cc:1939] T 4ff163c36c6d42238cce2396102f9a68 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:51.766073 6046 raft_consensus.cc:348] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.766870 6046 raft_consensus.cc:374] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:51.767155 6046 raft_consensus.cc:729] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:51.767865 6046 consensus_queue.cc:260] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.769675 6046 ts_tablet_manager.cc:1428] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 13:35:51.781415 6026 tablet_bootstrap.cc:492] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:51.782043 6026 ts_tablet_manager.cc:1397] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.293s user 0.010s sys 0.008s
I20260808 13:35:51.782397 6046 tablet_bootstrap.cc:492] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:51.787261 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_e4b2b77a6c9343b39de382dcd433c9e7 [id=6fdd797fbbf94dafa4f0048d8f4fb099]
W20260808 13:35:51.787513 5551 leader_election.cc:341] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.787983 5551 leader_election.cc:302] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:51.788686 8524 raft_consensus.cc:2726] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:51.788795 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_4d655ec59e044877882b5cbb4a10a597 [id=716d7d399d82408fb07943a0e0715f88]
I20260808 13:35:51.790280 6026 raft_consensus.cc:348] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.791078 6026 raft_consensus.cc:374] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:51.803845 8660 raft_consensus.cc:482] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.804462 8660 raft_consensus.cc:504] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:51.807539 8660 leader_election.cc:288] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:51.797145 5551 leader_election.cc:341] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.813138 5551 leader_election.cc:302] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:51.814589 5551 leader_election.cc:341] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.814831 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04d3dd028da4e7f86689e819548d0c5" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:51.814980 5551 leader_election.cc:302] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:51.816058 8613 raft_consensus.cc:2726] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:51.816589 8613 raft_consensus.cc:2726] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:51.817673 6026 raft_consensus.cc:729] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:51.819216 6026 consensus_queue.cc:260] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.821043 8652 leader_election.cc:288] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:51.823908 6026 ts_tablet_manager.cc:1428] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.041s user 0.005s sys 0.000s
I20260808 13:35:51.823706 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d31516006bc4a65b496dcf363db464a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:51.810072 8700 ts_tablet_manager.cc:1918] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:51.830988 5640 raft_consensus.cc:3037] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:51.831074 8700 log.cc:1196] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/4ff163c36c6d42238cce2396102f9a68
I20260808 13:35:51.832231 8700 ts_tablet_manager.cc:1939] T 4ff163c36c6d42238cce2396102f9a68 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:51.835409 6026 tablet_bootstrap.cc:492] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:51.835423 4820 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.170.61:46555 timed out after 0.728s (SENT)
I20260808 13:35:51.838825 5640 raft_consensus.cc:2454] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:51.840682 6033 tablet_bootstrap.cc:654] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:51.840417 5772 leader_election.cc:302] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:51.847302 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_4579204171cf47c6b668cedc2efaa649 [id=71918a8fb02640a48c7838e607490743]
I20260808 13:35:51.848222 8660 raft_consensus.cc:2781] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:51.848726 8660 raft_consensus.cc:686] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:51.849000 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d31516006bc4a65b496dcf363db464a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:51.849573 8660 consensus_queue.cc:237] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.854851 8656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:35:51.855468 8656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:51.855930 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_0616044c71e04d1a83b3ffd25b84f54e [id=71c485130d814412bd042eb5012ba511]
W20260808 13:35:51.868744 5771 leader_election.cc:341] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.880318 6026 tablet_bootstrap.cc:654] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:51.894820 8613 raft_consensus.cc:482] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.895905 8654 raft_consensus.cc:482] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:51.895630 8613 raft_consensus.cc:504] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.896672 8654 raft_consensus.cc:504] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:51.899868 8654 leader_election.cc:288] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:51.902456 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 4ff163c36c6d42238cce2396102f9a68 (table default.Table_00b62ebc438b40209bd662e8cd04fb29 [id=e90bcbe52e7b4ad1b60f0324fe158df4]) successfully deleted
I20260808 13:35:51.903959 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007c7222c7bc48329420c740b31ddfae" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:51.907411 5550 leader_election.cc:341] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:51.906220 8613 leader_election.cc:288] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
W20260808 13:35:51.894120 7662 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.3:55463 (request call id 38) took 1350 ms (client timeout 1000 ms). Trace:
W20260808 13:35:51.913515 7662 rpcz_store.cc:269] 0808 13:35:50.543524 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:50.543818 (+ 294us) service_pool.cc:227] Handling call
0808 13:35:50.558182 (+ 14364us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b54000303a8. stack: 00007ff0fbb8b033 00007ff0fbb8af30 00007ff0fb70c534 00000000004db2b4 00000000004db249 00000000004db0e1 00000000004da6bc 00007ff1049e6d54 00007ff104ae0c3e 00007ff102617fb4 00007ff102617f42 00007ff102617e71 00007ff102617dec 00007ff1026170a2 00007ff1008c194c 00007ff1008c0d86
0808 13:35:51.854393 (+1296211us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400030554. stack: 00007ff0fbb8b033 00007ff0fbb8af30 00007ff0fb70c534 00000000004db2b4 00000000004db249 00007ff102a97af0 00007ff102b1af2a 00007ff1049e6d6c 00007ff104ae0c3e 00007ff102617fb4 00007ff102617f42 00007ff102617e71 00007ff102617dec 00007ff1026170a2 00007ff1008c194c 00007ff1008c0d86
0808 13:35:51.893393 (+ 39000us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3003921536}
I20260808 13:35:51.915879 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04d3dd028da4e7f86689e819548d0c5" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:51.874536 7659 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:44803 (request call id 44) took 1398 ms (client timeout 1000 ms). Trace:
I20260808 13:35:51.903021 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d31516006bc4a65b496dcf363db464a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:51.917915 7659 rpcz_store.cc:269] 0808 13:35:50.476325 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:50.477780 (+ 1455us) service_pool.cc:227] Handling call
0808 13:35:51.869932 (+1392152us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400030554. stack: 00007ff0fbb8b033 00007ff0fbb8af30 00007ff0fb70c534 00000000004db2b4 00000000004db249 00007ff102a97af0 00007ff102b1af2a 00007ff1049e6d6c 00007ff104ae0c3e 00007ff102617fb4 00007ff102617f42 00007ff102617e71 00007ff102617dec 00007ff1026170a2 00007ff1008c194c 00007ff1008c0d86
0808 13:35:51.874015 (+ 4083us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3147671808}
I20260808 13:35:51.929014 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:51.932260 8698 tablet_replica.cc:330] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:51.938668 5630 raft_consensus.cc:2454] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:51.886932 8598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c875fc5bcd41a9b1bcf3bd4ae321d7. Latest consensus state: current_term: 4 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:35:51.940887 8598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:35:51.941366 5772 leader_election.cc:302] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:51.943562 8654 raft_consensus.cc:2781] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:51.944077 8654 raft_consensus.cc:482] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:51.944633 8654 raft_consensus.cc:3037] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:51.953675 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:51.952267 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_e4f01562a52f4079b8b916b057cf8014 [id=76320b3dd2994b4587dc8a6b3a71825d]
W20260808 13:35:51.966409 7658 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.1:49557 (request call id 45) took 1570 ms (client timeout 1000 ms). Trace:
W20260808 13:35:51.966856 7658 rpcz_store.cc:269] 0808 13:35:50.339642 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:50.339878 (+ 236us) service_pool.cc:227] Handling call
0808 13:35:51.890633 (+1550755us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400030554. stack: 00007ff0fbb8b033 00007ff0fbb8af30 00007ff0fb70c534 00000000004db2b4 00000000004db249 00007ff102a97af0 00007ff102b1af2a 00007ff1049e6d6c 00007ff104ae0c3e 00007ff102617fb4 00007ff102617f42 00007ff102617e71 00007ff102617dec 00007ff1026170a2 00007ff1008c194c 00007ff1008c0d86
0808 13:35:51.910201 (+ 19568us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3516662144}
I20260808 13:35:51.975867 8698 raft_consensus.cc:2229] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:51.976855 8698 raft_consensus.cc:2258] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260808 13:35:51.974676 8235 log.cc:924] Time spent T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5: Append to log took a long time: real 0.183s user 0.002s sys 0.000s
I20260808 13:35:51.989791 6046 tablet_bootstrap.cc:654] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:51.986394 8656 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:51.976047 7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36336 (request call id 3) took 1730 ms. Trace:
I20260808 13:35:52.001019 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_1c113a710bb14973be6ef54f41e550f8 [id=7929b3dde23940a8a96690fff0077013]
I20260808 13:35:52.003036 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_2269d7b97e6a488b9a86e8240b3b197e [id=7a78c3e7fd434fe48e5d0d8faa195f6e]
I20260808 13:35:52.005846 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:52.010674 5771 leader_election.cc:302] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:52.011791 8738 raft_consensus.cc:2726] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:52.012979 8738 raft_consensus.cc:482] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:52.013620 8738 raft_consensus.cc:504] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.015399 8698 ts_tablet_manager.cc:1905] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:52.017227 8738 leader_election.cc:288] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:52.018127 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bed5e6d1aa49a9b14ba86403c985ce" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:52.018836 5630 raft_consensus.cc:2454] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:52.020056 5772 leader_election.cc:302] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:52.021159 8738 raft_consensus.cc:2781] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:52.021622 8738 raft_consensus.cc:482] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:52.022070 8738 raft_consensus.cc:3037] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:52.027869 5771 leader_election.cc:341] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:52.040822 4820 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 205397us
I20260808 13:35:52.047518 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 4ff163c36c6d42238cce2396102f9a68 (table default.Table_00b62ebc438b40209bd662e8cd04fb29 [id=e90bcbe52e7b4ad1b60f0324fe158df4]) successfully deleted
I20260808 13:35:52.051347 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_c23b9f32680f412fbc697e136022e69c [id=7b7b344f5dda412389a0223f88351783]
W20260808 13:35:52.065327 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 369a942248bd4cec9ee9170fe55601a3 because the tablet was not found. No further retry: Not found: Tablet not found: 369a942248bd4cec9ee9170fe55601a3
W20260808 13:35:52.044340 4839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:55071 (request call id 58) took 3829 ms (client timeout 1000 ms). Trace:
I20260808 13:35:52.053686 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_432ac67683bd4fb4a2db65e5369a98c2 [id=7cf1c563f7f74ddaa65c0f335b1a3579]
W20260808 13:35:52.066355 4839 rpcz_store.cc:269] 0808 13:35:48.215286 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:48.215422 (+ 136us) service_pool.cc:227] Handling call
0808 13:35:51.614404 (+3398982us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.614423 (+ 19us) write_op.cc:454] Released schema lock
0808 13:35:51.754049 (+139626us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.754071 (+ 22us) write_op.cc:454] Released schema lock
0808 13:35:52.044264 (+290193us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:51.614230 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:51.614261 (+ 31us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.614279 (+ 18us) write_op.cc:454] Released schema lock
0808 13:35:51.614297 (+ 18us) write_op.cc:335] FINISH: Op aborted
Related trace 'op':
0808 13:35:51.749987 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 3e672d7386144b299688abea9c64854c is not leader of this config: current role FOLLOWER)
0808 13:35:51.750022 (+ 35us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:35:51.753844 (+ 3822us) write_op.cc:454] Released schema lock
0808 13:35:51.753876 (+ 32us) write_op.cc:335] FINISH: Op aborted
Metrics: {"reported_tablets":40,"child_traces":[["op",{}],["op",{}]]}
I20260808 13:35:52.068871 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_025eb6ef4e024bec917c3272ba1b8c35 [id=7dda6de1004344a987b5ede426b5a758]
W20260808 13:35:52.075116 5771 leader_election.cc:341] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.079278 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007c7222c7bc48329420c740b31ddfae" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:52.062462 8654 raft_consensus.cc:504] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.083771 8654 leader_election.cc:288] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:52.074383 5285 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 173571us
I20260808 13:35:52.084895 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:52.085423 5630 raft_consensus.cc:3037] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:52.109651 5630 raft_consensus.cc:2454] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
W20260808 13:35:52.110834 5283 consensus_peers.cc:487] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:52.120033 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_b84003117fd04b17930f7a3ba4e2d05f [id=7f4fa75c41e542928b94ef51cfd2f33a]
I20260808 13:35:52.122267 8738 raft_consensus.cc:504] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.134228 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_01763d5cef124bad885eaaba0686787b [id=8269e8c2637f41859e8ead761bc2d7bc]
I20260808 13:35:52.093164 6033 tablet_bootstrap.cc:492] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:52.128711 5772 leader_election.cc:302] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:52.138585 6033 ts_tablet_manager.cc:1397] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.500s user 0.008s sys 0.003s
I20260808 13:35:52.148649 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bed5e6d1aa49a9b14ba86403c985ce" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:52.149278 5630 raft_consensus.cc:3037] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:52.141230 8654 raft_consensus.cc:2781] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:52.154140 8654 raft_consensus.cc:686] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:52.155067 8654 consensus_queue.cc:237] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:52.156729 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 45378.2us
I20260808 13:35:52.159211 8627 raft_consensus.cc:482] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:52.159723 8627 raft_consensus.cc:504] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.161942 8627 leader_election.cc:288] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:52.165195 8738 leader_election.cc:288] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:52.140998 6033 raft_consensus.cc:348] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.166682 6033 raft_consensus.cc:374] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:52.169709 6033 raft_consensus.cc:729] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:52.171756 6033 consensus_queue.cc:260] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.174579 6033 ts_tablet_manager.cc:1428] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.036s user 0.004s sys 0.001s
I20260808 13:35:52.180362 6026 tablet_bootstrap.cc:492] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:52.180845 6026 ts_tablet_manager.cc:1397] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.346s user 0.011s sys 0.000s
I20260808 13:35:52.184332 8613 raft_consensus.cc:482] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:52.184739 8613 raft_consensus.cc:504] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.185532 5630 raft_consensus.cc:2454] T 96bed5e6d1aa49a9b14ba86403c985ce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:52.187456 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_920e3ee624f44ee4b19c15e0d7fb4358 [id=844dc96c93564895911d827c23c02689]
I20260808 13:35:52.187412 5772 leader_election.cc:302] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:52.187891 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d12106f0d794a79935999a2b8dbe7e9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:52.189339 5550 leader_election.cc:341] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.186553 8613 leader_election.cc:288] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:52.190878 6046 tablet_bootstrap.cc:492] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:52.191496 6046 ts_tablet_manager.cc:1397] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.409s user 0.005s sys 0.009s
I20260808 13:35:52.194537 5880 raft_consensus.cc:3037] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:52.195793 5877 raft_consensus.cc:1261] T 023fe27fc8ce4321a43a1f6037dd9cce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 3 FOLLOWER]: Refusing update from remote peer 946c9d8462644deba5b45fa724f1923c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260808 13:35:52.195729 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bed5e6d1aa49a9b14ba86403c985ce" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:52.197105 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e36d643850411498c23699835f55a6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:52.198124 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:52.205078 8627 consensus_queue.cc:1035] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260808 13:35:52.114370 8684 ts_tablet_manager.cc:1918] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:52.206275 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_6eaa73a8f6204e05808ac7d1b397ec04 [id=84e585111c464794ac7ee0e88227ee7a]
I20260808 13:35:52.206192 8684 log.cc:1196] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/86c2b34c95244472beec43643e60ff5f
I20260808 13:35:52.207252 8684 ts_tablet_manager.cc:1939] T 86c2b34c95244472beec43643e60ff5f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:52.202108 8654 raft_consensus.cc:2781] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
W20260808 13:35:52.211879 5283 leader_election.cc:341] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:52.213783 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 3a27517560d64a3b8da2cf55980e18a8 because the tablet was not found. No further retry: Not found: Tablet not found: 3a27517560d64a3b8da2cf55980e18a8
I20260808 13:35:52.215057 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_1955ad58908247a3b3002dc6a8ff2336 [id=85520e0aaa3942268225520f28506cd8]
W20260808 13:35:52.215205 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 851a0b374a854efd87dd66e39b9df2d5 because the tablet was not found. No further retry: Not found: Tablet not found: 851a0b374a854efd87dd66e39b9df2d5
I20260808 13:35:52.216320 6033 tablet_bootstrap.cc:492] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:52.216456 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet daa59a18ae624bc8975bee2c460895a7 because the tablet was not found. No further retry: Not found: Tablet not found: daa59a18ae624bc8975bee2c460895a7
I20260808 13:35:52.216595 8700 tablet_replica.cc:330] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
W20260808 13:35:52.217000 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 362dbd6990e740b39690537af98aa2f8 because the tablet was not found. No further retry: Not found: Tablet not found: 362dbd6990e740b39690537af98aa2f8
W20260808 13:35:52.217547 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 86c2b34c95244472beec43643e60ff5f because the tablet was not found. No further retry: Not found: Tablet not found: 86c2b34c95244472beec43643e60ff5f
W20260808 13:35:52.218401 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 5c0b76c4fdd94a6e9d9bb7709ff1b4be because the tablet was not found. No further retry: Not found: Tablet not found: 5c0b76c4fdd94a6e9d9bb7709ff1b4be
I20260808 13:35:52.218581 8700 raft_consensus.cc:2229] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:52.219488 8700 raft_consensus.cc:2258] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:52.221346 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e36d643850411498c23699835f55a6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:52.221951 5630 raft_consensus.cc:2454] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:52.223901 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_db29cf6251b947269c7a48cfc05ea2e5 [id=86e7a76979764ac09c964217ebe7fae4]
I20260808 13:35:52.235730 6033 tablet_bootstrap.cc:654] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:52.239930 8613 raft_consensus.cc:482] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:52.240594 8613 raft_consensus.cc:504] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.241758 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_d99de9b6e1e64b48873233ac5849ea09 [id=88fdd9e2283a4018ad2e1d02dc9cd4ea]
I20260808 13:35:52.244776 8613 leader_election.cc:288] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:52.244943 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09eb729339e47eb8db88eb6509daab4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:52.247764 6026 raft_consensus.cc:348] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.248647 6026 raft_consensus.cc:374] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:52.248972 6026 raft_consensus.cc:729] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
W20260808 13:35:52.250841 5550 leader_election.cc:341] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.210377 8654 raft_consensus.cc:686] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:52.252609 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bed5e6d1aa49a9b14ba86403c985ce" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:52.254004 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf]
I20260808 13:35:52.249967 8700 ts_tablet_manager.cc:1905] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:52.264720 8627 consensus_queue.cc:1035] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:52.268102 8684 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 474) took 1052 ms. Trace:
I20260808 13:35:52.268622 8684 rpcz_store.cc:276] 0808 13:35:51.215771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.216158 (+ 387us) service_pool.cc:227] Handling call
0808 13:35:51.423041 (+206883us) ts_tablet_manager.cc:1145] Deleting tablet 86c2b34c95244472beec43643e60ff5f
0808 13:35:51.508752 (+ 85711us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:51.810087 (+301335us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:52.114124 (+304037us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:52.268035 (+153911us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":79814,"tablet-delete.queue_time_us":206237,"thread_start_us":260256,"threads_started":1}
I20260808 13:35:52.269413 8743 consensus_queue.cc:1035] T b8e3245c8ba54f69bc329a5b75115127 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:52.270217 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 86c2b34c95244472beec43643e60ff5f (table default.Table_2df92438979447cc89b7a5301a13d6a9 [id=40cafe491872437cba9d7294fff5519a]) successfully deleted
I20260808 13:35:52.271201 8684 tablet_replica.cc:330] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:52.272267 5285 leader_election.cc:302] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:52.274375 8740 raft_consensus.cc:2781] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:52.274787 8740 raft_consensus.cc:482] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:52.275139 8740 raft_consensus.cc:3037] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:52.285290 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_a8ff8993913d4a05a3d8ab3bb339b7ac [id=8ac0944cbe564a949c732ec146a9cf94]
I20260808 13:35:52.287185 8613 raft_consensus.cc:482] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:52.287587 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_62ad61e62a5b49b480ec08bc990efa85 [id=8e1d239d7f7446fa89184bf11ddb88d9]
I20260808 13:35:52.287715 8613 raft_consensus.cc:504] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:52.288856 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_a16be245c9864b969707b18ee86278d0 [id=8f151f8823464520ae72942a6c79d8bb]
I20260808 13:35:52.290108 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_2d152c1c176a4bc58ff1f37b2a9bfd0c [id=8f74146d8e454203a91ce42acd6a5233]
I20260808 13:35:52.291983 8613 leader_election.cc:288] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:52.296942 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 84436.4us
I20260808 13:35:52.304522 5630 raft_consensus.cc:1261] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 3 FOLLOWER]: Refusing update from remote peer 946c9d8462644deba5b45fa724f1923c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260808 13:35:52.305124 5630 pending_rounds.cc:76] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb: Aborting all ops after (but not including) 1
I20260808 13:35:52.305449 5630 pending_rounds.cc:97] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260808 13:35:52.305887 5630 raft_consensus.cc:2866] T 023fe27fc8ce4321a43a1f6037dd9cce P 5cd987c3dbdb4168beb73cd7e577c7cb [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260808 13:35:52.315034 8740 raft_consensus.cc:504] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:52.316548 5551 leader_election.cc:341] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.316963 5551 leader_election.cc:302] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:52.318133 8613 raft_consensus.cc:2726] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:52.291477 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9e8d48c55349058cf80bf7c2e149e7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:52.323943 5550 leader_election.cc:341] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.325781 8684 raft_consensus.cc:2229] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:52.330286 8684 raft_consensus.cc:2258] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:52.279958 8654 consensus_queue.cc:237] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.348654 6033 tablet_bootstrap.cc:492] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:52.349117 6033 ts_tablet_manager.cc:1397] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.133s user 0.004s sys 0.012s
I20260808 13:35:52.351486 6033 raft_consensus.cc:348] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.335407 8684 ts_tablet_manager.cc:1905] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:52.353979 5771 leader_election.cc:341] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.000073 7736 rpcz_store.cc:276] 0808 13:35:50.245347 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:50.246084 (+ 737us) service_pool.cc:227] Handling call
0808 13:35:50.304601 (+ 58517us) raft_consensus.cc:1457] Updating replica for 106 ops
0808 13:35:50.335011 (+ 30410us) raft_consensus.cc:1520] Early marking committed up to index 0
0808 13:35:50.335072 (+ 61us) raft_consensus.cc:1525] Triggering prepare for 106 ops
0808 13:35:50.337393 (+ 2321us) write_op.cc:270] Start()
0808 13:35:50.337589 (+ 196us) write_op.cc:276] Timestamp: P: 1786196138184409 usec, L: 0
0808 13:35:50.338646 (+ 1057us) write_op.cc:270] Start()
0808 13:35:50.338827 (+ 181us) write_op.cc:276] Timestamp: P: 1786196138188131 usec, L: 0
0808 13:35:50.339726 (+ 899us) write_op.cc:270] Start()
0808 13:35:50.339866 (+ 140us) write_op.cc:276] Timestamp: P: 1786196138226857 usec, L: 0
0808 13:35:50.340814 (+ 948us) write_op.cc:270] Start()
0808 13:35:50.341027 (+ 213us) write_op.cc:276] Timestamp: P: 1786196138244190 usec, L: 0
0808 13:35:50.342076 (+ 1049us) write_op.cc:270] Start()
0808 13:35:50.342246 (+ 170us) write_op.cc:276] Timestamp: P: 1786196138353054 usec, L: 0
0808 13:35:50.348945 (+ 6699us) write_op.cc:270] Start()
0808 13:35:50.349119 (+ 174us) write_op.cc:276] Timestamp: P: 1786196138361286 usec, L: 0
0808 13:35:50.349936 (+ 817us) write_op.cc:270] Start()
0808 13:35:50.350142 (+ 206us) write_op.cc:276] Timestamp: P: 1786196138369864 usec, L: 0
0808 13:35:50.357445 (+ 7303us) write_op.cc:270] Start()
0808 13:35:50.357596 (+ 151us) write_op.cc:276] Timestamp: P: 1786196138525736 usec, L: 0
0808 13:35:50.362185 (+ 4589us) write_op.cc:270] Start()
0808 13:35:50.362369 (+ 184us) write_op.cc:276] Timestamp: P: 1786196138832152 usec, L: 0
0808 13:35:50.367811 (+ 5442us) write_op.cc:270] Start()
0808 13:35:50.367933 (+ 122us) write_op.cc:276] Timestamp: P: 1786196139214062 usec, L: 0
0808 13:35:50.368737 (+ 804us) write_op.cc:270] Start()
0808 13:35:50.368909 (+ 172us) write_op.cc:276] Timestamp: P: 1786196139258229 usec, L: 0
0808 13:35:50.369768 (+ 859us) write_op.cc:270] Start()
0808 13:35:50.369931 (+ 163us) write_op.cc:276] Timestamp: P: 1786196139286152 usec, L: 0
0808 13:35:50.370894 (+ 963us) write_op.cc:270] Start()
0808 13:35:50.371007 (+ 113us) write_op.cc:276] Timestamp: P: 1786196139290464 usec, L: 0
0808 13:35:50.372400 (+ 1393us) write_op.cc:270] Start()
0808 13:35:50.372541 (+ 141us) write_op.cc:276] Timestamp: P: 1786196139444803 usec, L: 0
0808 13:35:50.373246 (+ 705us) write_op.cc:270] Start()
0808 13:35:50.373442 (+ 196us) write_op.cc:276] Timestamp: P: 1786196139457997 usec, L: 0
0808 13:35:50.377683 (+ 4241us) write_op.cc:270] Start()
0808 13:35:50.378564 (+ 881us) write_op.cc:276] Timestamp: P: 1786196139515528 usec, L: 0
0808 13:35:50.379684 (+ 1120us) write_op.cc:270] Start()
0808 13:35:50.379858 (+ 174us) write_op.cc:276] Timestamp: P: 1786196139609311 usec, L: 0
0808 13:35:50.380783 (+ 925us) write_op.cc:270] Start()
0808 13:35:50.380966 (+ 183us) write_op.cc:276] Timestamp: P: 1786196139632153 usec, L: 0
0808 13:35:50.479458 (+ 98492us) write_op.cc:270] Start()
0808 13:35:50.479801 (+ 343us) write_op.cc:276] Timestamp: P: 1786196139634119 usec, L: 0
0808 13:35:50.491973 (+ 12172us) write_op.cc:270] Start()
0808 13:35:50.492159 (+ 186us) write_op.cc:276] Timestamp: P: 1786196139769541 usec, L: 0
0808 13:35:50.492937 (+ 778us) write_op.cc:270] Start()
0808 13:35:50.493047 (+ 110us) write_op.cc:276] Timestamp: P: 1786196139811624 usec, L: 0
0808 13:35:50.493609 (+ 562us) write_op.cc:270] Start()
0808 13:35:50.493720 (+ 111us) write_op.cc:276] Timestamp: P: 1786196139873797 usec, L: 0
0808 13:35:50.494246 (+ 526us) write_op.cc:270] Start()
0808 13:35:50.494330 (+ 84us) write_op.cc:276] Timestamp: P: 1786196139917737 usec, L: 0
0808 13:35:50.501348 (+ 7018us) write_op.cc:270] Start()
0808 13:35:50.501471 (+ 123us) write_op.cc:276] Timestamp: P: 1786196139938413 usec, L: 0
0808 13:35:50.502576 (+ 1105us) write_op.cc:270] Start()
0808 13:35:50.510811 (+ 8235us) write_op.cc:276] Timestamp: P: 1786196139967561 usec, L: 0
0808 13:35:50.511479 (+ 668us) write_op.cc:270] Start()
0808 13:35:50.511581 (+ 102us) write_op.cc:276] Timestamp: P: 1786196139989628 usec, L: 0
0808 13:35:50.514018 (+ 2437us) write_op.cc:270] Start()
0808 13:35:50.514113 (+ 95us) write_op.cc:276] Timestamp: P: 1786196139993037 usec, L: 0
0808 13:35:50.514627 (+ 514us) write_op.cc:270] Start()
0808 13:35:50.514706 (+ 79us) write_op.cc:276] Timestamp: P: 1786196140002364 usec, L: 0
0808 13:35:50.515162 (+ 456us) write_op.cc:270] Start()
0808 13:35:50.515259 (+ 97us) write_op.cc:276] Timestamp: P: 1786196140146846 usec, L: 0
0808 13:35:50.515718 (+ 459us) write_op.cc:270] Start()
0808 13:35:50.515795 (+ 77us) write_op.cc:276] Timestamp: P: 1786196140173331 usec, L: 0
0808 13:35:50.516280 (+ 485us) write_op.cc:270] Start()
0808 13:35:50.516364 (+ 84us) write_op.cc:276] Timestamp: P: 1786196140176137 usec, L: 0
0808 13:35:50.516805 (+ 441us) write_op.cc:270] Start()
0808 13:35:50.516905 (+ 100us) write_op.cc:276] Timestamp: P: 1786196140365427 usec, L: 0
0808 13:35:50.517510 (+ 605us) write_op.cc:270] Start()
0808 13:35:50.517602 (+ 92us) write_op.cc:276] Timestamp: P: 1786196140488226 usec, L: 0
0808 13:35:50.518083 (+ 481us) write_op.cc:270] Start()
0808 13:35:50.518162 (+ 79us) write_op.cc:276] Timestamp: P: 1786196140501104 usec, L: 0
0808 13:35:50.575056 (+ 56894us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b54000303a8. stack: 00007ff0fbb8b033 00007ff0fbb8af30 00007ff0fb70c534 00000000004db2b4 00000000004db249 00007ff103a547f1 00007ff103a7b204 00007ff102f6da9b 00007ff102f34f96 00007ff102f34acf 00007ff102b09040 00007ff102b12536 00007ff102b1199e 00007ff103aee2e9 00007ff1022c4684 00007ff1022c4612
0808 13:35:50.575120 (+ 64us) write_op.cc:270] Start()
0808 13:35:50.575238 (+ 118us) write_op.cc:276] Timestamp: P: 1786196140631301 usec, L: 0
0808 13:35:50.575900 (+ 662us) write_op.cc:270] Start()
0808 13:35:50.575997 (+ 97us) write_op.cc:276] Timestamp: P: 1786196140636925 usec, L: 0
0808 13:35:50.576498 (+ 501us) write_op.cc:270] Start()
0808 13:35:50.576579 (+ 81us) write_op.cc:276] Timestamp: P: 1786196140662759 usec, L: 0
0808 13:35:50.577048 (+ 469us) write_op.cc:270] Start()
0808 13:35:50.577124 (+ 76us) write_op.cc:276] Timestamp: P: 1786196140699150 usec, L: 0
0808 13:35:50.577615 (+ 491us) write_op.cc:270] Start()
0808 13:35:50.577698 (+ 83us) write_op.cc:276] Timestamp: P: 1786196140912483 usec, L: 0
0808 13:35:50.578177 (+ 479us) write_op.cc:270] Start()
0808 13:35:50.578277 (+ 100us) write_op.cc:276] Timestamp: P: 1786196140918118 usec, L: 0
0808 13:35:50.629289 (+ 51012us) write_op.cc:270] Start()
0808 13:35:50.629527 (+ 238us) write_op.cc:276] Timestamp: P: 1786196140941501 usec, L: 0
0808 13:35:50.644597 (+ 15070us) write_op.cc:270] Start()
0808 13:35:50.644739 (+ 142us) write_op.cc:276] Timestamp: P: 1786196140952301 usec, L: 0
0808 13:35:50.645585 (+ 846us) write_op.cc:270] Start()
0808 13:35:50.645691 (+ 106us) write_op.cc:276] Timestamp: P: 1786196140954296 usec, L: 0
0808 13:35:50.646293 (+ 602us) write_op.cc:270] Start()
0808 13:35:50.646391 (+ 98us) write_op.cc:276] Timestamp: P: 1786196141053607 usec, L: 0
0808 13:35:50.646967 (+ 576us) write_op.cc:270] Start()
0808 13:35:50.647066 (+ 99us) write_op.cc:276] Timestamp: P: 1786196141115299 usec, L: 0
0808 13:35:50.647552 (+ 486us) write_op.cc:270] Start()
0808 13:35:50.647645 (+ 93us) write_op.cc:276] Timestamp: P: 1786196141135919 usec, L: 0
0808 13:35:50.692167 (+ 44522us) write_op.cc:270] Start()
0808 13:35:50.692300 (+ 133us) write_op.cc:276] Timestamp: P: 1786196141170827 usec, L: 0
0808 13:35:50.693023 (+ 723us) write_op.cc:270] Start()
0808 13:35:50.693107 (+ 84us) write_op.cc:276] Timestamp: P: 1786196141230655 usec, L: 0
0808 13:35:50.693589 (+ 482us) write_op.cc:270] Start()
0808 13:35:50.693690 (+ 101us) write_op.cc:276] Timestamp: P: 1786196141239364 usec, L: 0
0808 13:35:50.744291 (+ 50601us) write_op.cc:270] Start()
0808 13:35:50.744413 (+ 122us) write_op.cc:276] Timestamp: P: 1786196141241176 usec, L: 0
0808 13:35:50.745063 (+ 650us) write_op.cc:270] Start()
0808 13:35:50.745156 (+ 93us) write_op.cc:276] Timestamp: P: 1786196141259634 usec, L: 0
0808 13:35:50.799870 (+ 54714us) write_op.cc:270] Start()
0808 13:35:50.799975 (+ 105us) write_op.cc:276] Timestamp: P: 1786196141311303 usec, L: 0
0808 13:35:50.800576 (+ 601us) write_op.cc:270] Start()
0808 13:35:50.800685 (+ 109us) write_op.cc:276] Timestamp: P: 1786196141324372 usec, L: 0
0808 13:35:50.810113 (+ 9428us) write_op.cc:270] Start()
0808 13:35:50.810211 (+ 98us) write_op.cc:276] Timestamp: P: 1786196141335356 usec, L: 0
0808 13:35:50.810847 (+ 636us) write_op.cc:270] Start()
0808 13:35:50.810947 (+ 100us) write_op.cc:276] Timestamp: P: 1786196141431955 usec, L: 0
0808 13:35:50.811547 (+ 600us) write_op.cc:270] Start()
0808 13:35:50.811648 (+ 101us) write_op.cc:276] Timestamp: P: 1786196141434023 usec, L: 0
0808 13:35:50.827076 (+ 15428us) write_op.cc:270] Start()
0808 13:35:50.827275 (+ 199us) write_op.cc:276] Timestamp: P: 1786196141494390 usec, L: 0
0808 13:35:50.827844 (+ 569us) write_op.cc:270] Start()
0808 13:35:50.827939 (+ 95us) write_op.cc:276] Timestamp: P: 1786196141496868 usec, L: 0
0808 13:35:50.828412 (+ 473us) write_op.cc:270] Start()
0808 13:35:50.828486 (+ 74us) write_op.cc:276] Timestamp: P: 1786196141685537 usec, L: 0
0808 13:35:50.828954 (+ 468us) write_op.cc:270] Start()
0808 13:35:50.829041 (+ 87us) write_op.cc:276] Timestamp: P: 1786196141689050 usec, L: 0
0808 13:35:50.829547 (+ 506us) write_op.cc:270] Start()
0808 13:35:50.829636 (+ 89us) write_op.cc:276] Timestamp: P: 1786196141731937 usec, L: 0
0808 13:35:50.869801 (+ 40165us) write_op.cc:270] Start()
0808 13:35:50.869909 (+ 108us) write_op.cc:276] Timestamp: P: 1786196141740824 usec, L: 0
0808 13:35:50.870577 (+ 668us) write_op.cc:270] Start()
0808 13:35:50.870686 (+ 109us) write_op.cc:276] Timestamp: P: 1786196141747830 usec, L: 0
0808 13:35:50.871210 (+ 524us) write_op.cc:270] Start()
0808 13:35:50.871305 (+ 95us) write_op.cc:276] Timestamp: P: 1786196141865119 usec, L: 0
0808 13:35:50.871902 (+ 597us) write_op.cc:270] Start()
0808 13:35:50.871986 (+ 84us) write_op.cc:276] Timestamp: P: 1786196141945298 usec, L: 0
0808 13:35:50.872465 (+ 479us) write_op.cc:270] Start()
0808 13:35:50.872550 (+ 85us) write_op.cc:276] Timestamp: P: 1786196142033579 usec, L: 0
0808 13:35:50.873014 (+ 464us) write_op.cc:270] Start()
0808 13:35:50.873093 (+ 79us) write_op.cc:276] Timestamp: P: 1786196142080328 usec, L: 0
0808 13:35:50.873579 (+ 486us) write_op.cc:270] Start()
0808 13:35:50.873663 (+ 84us) write_op.cc:276] Timestamp: P: 1786196142082150 usec, L: 0
0808 13:35:50.907580 (+ 33917us) write_op.cc:270] Start()
0808 13:35:50.907680 (+ 100us) write_op.cc:276] Timestamp: P: 1786196142123062 usec, L: 0
0808 13:35:50.917147 (+ 9467us) write_op.cc:270] Start()
0808 13:35:50.917253 (+ 106us) write_op.cc:276] Timestamp: P: 1786196142163960 usec, L: 0
0808 13:35:50.917892 (+ 639us) write_op.cc:270] Start()
0808 13:35:50.917978 (+ 86us) write_op.cc:276] Timestamp: P: 1786196142215971 usec, L: 0
0808 13:35:50.918466 (+ 488us) write_op.cc:270] Start()
0808 13:35:50.918543 (+ 77us) write_op.cc:276] Timestamp: P: 1786196142467451 usec, L: 0
0808 13:35:50.931327 (+ 12784us) write_op.cc:270] Start()
0808 13:35:50.931433 (+ 106us) write_op.cc:276] Timestamp: P: 1786196142545529 usec, L: 0
0808 13:35:50.931965 (+ 532us) write_op.cc:270] Start()
0808 13:35:50.932047 (+ 82us) write_op.cc:276] Timestamp: P: 1786196142595241 usec, L: 0
0808 13:35:50.932513 (+ 466us) write_op.cc:270] Start()
0808 13:35:50.932596 (+ 83us) write_op.cc:276] Timestamp: P: 1786196142683463 usec, L: 0
0808 13:35:50.933100 (+ 504us) write_op.cc:270] Start()
0808 13:35:50.933187 (+ 87us) write_op.cc:276] Timestamp: P: 1786196142685772 usec, L: 0
0808 13:35:50.933695 (+ 508us) write_op.cc:270] Start()
0808 13:35:50.933777 (+ 82us) write_op.cc:276] Timestamp: P: 1786196142832061 usec, L: 0
0808 13:35:50.934253 (+ 476us) write_op.cc:270] Start()
0808 13:35:50.934338 (+ 85us) write_op.cc:276] Timestamp: P: 1786196142837145 usec, L: 0
0808 13:35:50.959322 (+ 24984us) write_op.cc:270] Start()
0808 13:35:50.960835 (+ 1513us) write_op.cc:276] Timestamp: P: 1786196143056434 usec, L: 0
0808 13:35:50.961613 (+ 778us) write_op.cc:270] Start()
0808 13:35:50.961704 (+ 91us) write_op.cc:276] Timestamp: P: 1786196143427504 usec, L: 0
0808 13:35:50.966981 (+ 5277us) write_op.cc:270] Start()
0808 13:35:50.967079 (+ 98us) write_op.cc:276] Timestamp: P: 1786196143429916 usec, L: 0
0808 13:35:50.967594 (+ 515us) write_op.cc:270] Start()
0808 13:35:50.976358 (+ 8764us) write_op.cc:276] Timestamp: P: 1786196143900962 usec, L: 0
0808 13:35:51.002772 (+ 26414us) write_op.cc:270] Start()
0808 13:35:51.002885 (+ 113us) write_op.cc:276] Timestamp: P: 1786196144095133 usec, L: 0
0808 13:35:51.003539 (+ 654us) write_op.cc:270] Start()
0808 13:35:51.003777 (+ 238us) write_op.cc:276] Timestamp: P: 1786196144116795 usec, L: 0
0808 13:35:51.004291 (+ 514us) write_op.cc:270] Start()
0808 13:35:51.004374 (+ 83us) write_op.cc:276] Timestamp: P: 1786196144179115 usec, L: 0
0808 13:35:51.004821 (+ 447us) write_op.cc:270] Start()
0808 13:35:51.004902 (+ 81us) write_op.cc:276] Timestamp: P: 1786196144207767 usec, L: 0
0808 13:35:51.005329 (+ 427us) write_op.cc:270] Start()
0808 13:35:51.005404 (+ 75us) write_op.cc:276] Timestamp: P: 1786196144864569 usec, L: 0
0808 13:35:51.005883 (+ 479us) write_op.cc:270] Start()
0808 13:35:51.005960 (+ 77us) write_op.cc:276] Timestamp: P: 1786196145040095 usec, L: 0
0808 13:35:51.006441 (+ 481us) write_op.cc:270] Start()
0808 13:35:51.006526 (+ 85us) write_op.cc:276] Timestamp: P: 1786196145263433 usec, L: 0
0808 13:35:51.055606 (+ 49080us) write_op.cc:270] Start()
0808 13:35:51.055709 (+ 103us) write_op.cc:276] Timestamp: P: 1786196145456469 usec, L: 0
0808 13:35:51.056291 (+ 582us) write_op.cc:270] Start()
0808 13:35:51.056376 (+ 85us) write_op.cc:276] Timestamp: P: 1786196145634920 usec, L: 0
0808 13:35:51.056866 (+ 490us) write_op.cc:270] Start()
0808 13:35:51.056945 (+ 79us) write_op.cc:276] Timestamp: P: 1786196145848529 usec, L: 0
0808 13:35:51.057522 (+ 577us) write_op.cc:270] Start()
0808 13:35:51.057601 (+ 79us) write_op.cc:276] Timestamp: P: 1786196146108027 usec, L: 0
0808 13:35:51.058102 (+ 501us) write_op.cc:270] Start()
0808 13:35:51.058179 (+ 77us) write_op.cc:276] Timestamp: P: 1786196146139688 usec, L: 0
0808 13:35:51.076155 (+ 17976us) write_op.cc:270] Start()
0808 13:35:51.076277 (+ 122us) write_op.cc:276] Timestamp: P: 1786196146460199 usec, L: 0
0808 13:35:51.076881 (+ 604us) write_op.cc:270] Start()
0808 13:35:51.076974 (+ 93us) write_op.cc:276] Timestamp: P: 1786196146527989 usec, L: 0
0808 13:35:51.077470 (+ 496us) write_op.cc:270] Start()
0808 13:35:51.077555 (+ 85us) write_op.cc:276] Timestamp: P: 1786196146865235 usec, L: 0
0808 13:35:51.078007 (+ 452us) write_op.cc:270] Start()
0808 13:35:51.078085 (+ 78us) write_op.cc:276] Timestamp: P: 1786196147219507 usec, L: 0
0808 13:35:51.078542 (+ 457us) write_op.cc:270] Start()
0808 13:35:51.122684 (+ 44142us) write_op.cc:276] Timestamp: P: 1786196147311634 usec, L: 0
0808 13:35:51.123395 (+ 711us) write_op.cc:270] Start()
0808 13:35:51.123494 (+ 99us) write_op.cc:276] Timestamp: P: 1786196147528441 usec, L: 0
0808 13:35:51.124099 (+ 605us) write_op.cc:270] Start()
0808 13:35:51.124247 (+ 148us) write_op.cc:276] Timestamp: P: 1786196147622764 usec, L: 0
0808 13:35:51.124741 (+ 494us) write_op.cc:270] Start()
0808 13:35:51.124817 (+ 76us) write_op.cc:276] Timestamp: P: 1786196147719814 usec, L: 0
0808 13:35:51.125258 (+ 441us) write_op.cc:270] Start()
0808 13:35:51.125349 (+ 91us) write_op.cc:276] Timestamp: P: 1786196147796453 usec, L: 0
0808 13:35:51.125824 (+ 475us) write_op.cc:270] Start()
0808 13:35:51.125901 (+ 77us) write_op.cc:276] Timestamp: P: 1786196147890245 usec, L: 0
0808 13:35:51.166674 (+ 40773us) write_op.cc:270] Start()
0808 13:35:51.166795 (+ 121us) write_op.cc:276] Timestamp: P: 1786196147957218 usec, L: 0
0808 13:35:51.668960 (+502165us) log.cc:842] Serialized 64438 byte log entry
0808 13:35:51.704448 (+ 35488us) raft_consensus.cc:1631] Marking committed up to 594
0808 13:35:51.853690 (+149242us) raft_consensus.cc:1683] Filling consensus response to leader.
0808 13:35:51.875324 (+ 21634us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400030554. stack: 00007ff0fbb8b033 00007ff0fbb8af30 00007ff0fb70c534 00000000004db2b4 00000000004db249 00007ff102a97af0 00007ff102b1af2a 00007ff104b1230c 00007ff104b1a72c 00007ff104b1a6d8 00007ff104b1a658 00007ff104b1a613 00007ff104b198bc 00007ff102f4b0e6 00007ff102f4b078 00007ff102b312fa
0808 13:35:51.974825 (+ 99501us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0808 13:35:51.974986 (+ 161us) raft_consensus.cc:1671] finished
0808 13:35:51.974999 (+ 13us) raft_consensus.cc:1677] UpdateReplicas() finished
0808 13:35:51.975642 (+ 643us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:35:51.704607 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.704745 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.704850 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.704978 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.705106 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.705390 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.705541 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.705646 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.705795 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.705928 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.706037 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.706141 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.706257 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.706476 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.706573 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.731017 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.731126 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.731231 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.731325 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.731407 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.731682 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.731768 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.731854 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.731937 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.732024 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.732106 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.732187 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.732279 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.732367 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.732446 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.733053 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.733178 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.733276 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.733363 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.733450 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.733535 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.733637 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.733741 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.733846 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.733941 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.734038 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.734131 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.734241 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.734338 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.734433 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.734527 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.734636 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.734725 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.750974 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751077 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751159 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751252 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751348 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751429 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751510 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751588 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751671 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751746 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751828 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751908 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.751989 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.752065 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.752151 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.752237 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.752317 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.752394 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.752471 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.752556 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.752637 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.752735 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.763692 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.763841 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.763932 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.764006 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.782681 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.782815 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.782905 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.782997 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.783081 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.783160 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.830427 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.830550 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.830648 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.830734 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.851614 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.851726 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.851811 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.851969 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852064 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852142 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852247 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852327 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852410 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852487 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852568 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852654 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852765 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852863 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.852951 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.853049 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.853150 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.853250 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.853418 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.853521 (+ 0us) op_driver.cc:444] REPLICATION: finished
Related trace 'op':
0808 13:35:51.853626 (+ 0us) op_driver.cc:444] REPLICATION: finished
Metrics: {"raft.queue_time_us":7965,"raft.run_cpu_time_us":10293,"raft.run_wall_time_us":1598622,"spinlock_wait_cycles":3521920896,"thread_start_us":12593,"threads_started":1,"child_traces":[["op",{"replication_time_us":1367155}],["op",{"replication_time_us":1366089}],["op",{"replication_time_us":1365087}],["op",{"replication_time_us":1364130}],["op",{"replication_time_us":1362979}],["op",{"replication_time_us":1356388}],["op",{"replication_time_us":1355579}],["op",{"replication_time_us":1348159}],["op",{"mutex_wait_us":4591,"replication_time_us":1343550}],["op",{"replication_time_us":1338057}],["op",{"replication_time_us":1337248}],["op",{"replication_time_us":1336330}],["op",{"replication_time_us":1335322}],["op",{"replication_time_us":1334028}],["op",{"replication_time_us":1333293}],["op",{"replication_time_us":1353258}],["op",{"replication_time_us":1351408}],["op",{"replication_time_us":1350431}],["op",{"replication_time_us":1251886}],["op",{"replication_time_us":1239409}],["op",{"replication_time_us":1238701}],["op",{"replication_time_us":1238126}],["op",{"replication_time_us":1237588}],["op",{"replication_time_us":1230558}],["op",{"replication_time_us":1229522}],["op",{"replication_time_us":1220595}],["op",{"replication_time_us":1218139}],["op",{"replication_time_us":1217645}],["op",{"replication_time_us":1217168}],["op",{"replication_time_us":1216697}],["op",{"replication_time_us":1216239}],["op",{"replication_time_us":1216328}],["op",{"replication_time_us":1215738}],["op",{"replication_time_us":1215247}],["op",{"replication_time_us":1158299}],["op",{"replication_time_us":1157604}],["op",{"replication_time_us":1157098}],["op",{"replication_time_us":1156653}],["op",{"replication_time_us":1156192}],["op",{"replication_time_us":1155723}],["op",{"replication_time_us":1104709}],["op",{"replication_time_us":1089496}],["op",{"replication_time_us":1088600}],["op",{"replication_time_us":1088008}],["op",{"replication_time_us":1087430}],["op",{"replication_time_us":1086944}],["op",{"replication_time_us":1042420}],["op",{"replication_time_us":1041665}],["op",{"replication_time_us":1057319}],["op",{"replication_time_us":1006760}],["op",{"replication_time_us":1006061}],["op",{"replication_time_us":951347}],["op",{"replication_time_us":950736}],["op",{"replication_time_us":941281}],["op",{"replication_time_us":940634}],["op",{"replication_time_us":940012}],["op",{"replication_time_us":924566}],["op",{"replication_time_us":923873}],["op",{"replication_time_us":923390}],["op",{"replication_time_us":922920}],["op",{"replication_time_us":922406}],["op",{"replication_time_us":882233}],["op",{"replication_time_us":881533}],["op",{"replicat
W20260808 13:35:52.360448 5771 leader_election.cc:341] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:52.364398 5771 leader_election.cc:341] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.361776 6033 raft_consensus.cc:374] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:52.382851 6033 raft_consensus.cc:729] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:52.387791 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2]
I20260808 13:35:52.390061 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_d20dd87e2f844e739e8f18ec0be12963 [id=9477c15dbac14aa3a08e0b4a82097291]
I20260808 13:35:52.249712 6026 consensus_queue.cc:260] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.385391 6033 consensus_queue.cc:260] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.391726 8524 raft_consensus.cc:482] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:52.392208 8524 raft_consensus.cc:504] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.394215 8524 leader_election.cc:288] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:52.401978 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:52.405052 5550 leader_election.cc:341] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.417057 6033 ts_tablet_manager.cc:1428] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.068s user 0.007s sys 0.000s
I20260808 13:35:52.423499 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_5668bb8655e949f494b5c2e988ae33d5 [id=951c9bdf311a42a0b6fd001ef990e319]
I20260808 13:35:52.350232 8741 consensus_queue.cc:1035] T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260808 13:35:52.424930 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_938ca078ec5e4411b4be33056272983c [id=95f59bd3447c44c291270c448f53d78b]
I20260808 13:35:52.355937 8627 consensus_queue.cc:1035] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:52.355063 6046 raft_consensus.cc:348] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.384537 8698 ts_tablet_manager.cc:1918] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260808 13:35:52.441540 6046 raft_consensus.cc:374] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:52.441922 6046 raft_consensus.cc:729] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:52.441754 8698 log.cc:1196] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/7e8cf06462484c99a860762a82f7ad64
I20260808 13:35:52.443790 8698 ts_tablet_manager.cc:1939] T 7e8cf06462484c99a860762a82f7ad64 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:52.447525 8698 tablet_replica.cc:330] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:52.447638 8740 leader_election.cc:288] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:52.449090 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e36d643850411498c23699835f55a6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:52.454284 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_322ce0f117834a81978e588b16196c2a [id=9651f57557304d8eb476787530d19f39]
I20260808 13:35:52.454532 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e36d643850411498c23699835f55a6" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:52.455143 5644 raft_consensus.cc:3037] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:52.455768 6026 ts_tablet_manager.cc:1428] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.275s user 0.003s sys 0.002s
I20260808 13:35:52.456784 6026 tablet_bootstrap.cc:492] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:52.462906 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590]
I20260808 13:35:52.464570 5644 raft_consensus.cc:2454] T 42e36d643850411498c23699835f55a6 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:52.465421 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_9b62f270f0aa43bf92edae3c1e5b0ffd [id=99cf3d4080734790a0f96ac8b1239008]
I20260808 13:35:52.478670 6033 tablet_bootstrap.cc:492] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:52.488857 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_0fdee57de12549eab898e92f30fe691d [id=9ad354d775a84d32bd5eb627356e0a6b]
I20260808 13:35:52.495038 5880 raft_consensus.cc:3037] T a04d3dd028da4e7f86689e819548d0c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:52.503351 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_5398341b051f491ca73c85f9c8332eb9 [id=9dbadd28dd50468ba9a99548e008600f]
I20260808 13:35:52.517035 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_ce81522648e14e5daa8c8fb7d55ef7e4 [id=9df6ef937f0044ab9b0ded9876bead2d]
I20260808 13:35:52.527109 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_e4788552c1a8457a8d34951f6827c5e4 [id=a14f97b5af3741bebd7d88f5602675f1]
W20260808 13:35:52.530297 4823 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b6b421ff1f094d8689010b4ed6c3f7af (table default.Table_6eaa73a8f6204e05808ac7d1b397ec04 [id=84e585111c464794ac7ee0e88227ee7a], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b6b421ff1f094d8689010b4ed6c3f7af consensus state has no leader
I20260808 13:35:52.555842 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_280b5d6fc90b4748b675e41e2e7071f5 [id=a2a1f0d168074cabbd4a680e62535962]
I20260808 13:35:52.560695 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_9758d29f1a0b4ea5bceef3d5ce652ca1 [id=a3d0de3d513c43f29bea23a52883916a]
I20260808 13:35:52.566681 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_03cf1a01ebc14024a283b7e5ed420074 [id=a45370bd3bbc48279db129789b269480]
I20260808 13:35:52.566831 6033 tablet_bootstrap.cc:654] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:52.575099 7535 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.170.61:46555 timed out after 0.618s (SENT)
W20260808 13:35:52.578430 8756 log.cc:924] Time spent T 0756b5f24ed94a9fbe2a43efc7b8d40d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.161s user 0.001s sys 0.000s
I20260808 13:35:52.450333 8698 raft_consensus.cc:2229] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:52.585840 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a]
I20260808 13:35:52.586238 8698 raft_consensus.cc:2258] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:52.588742 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_9936d69a1f7b4853a7c37d5e8da2077f [id=a7e1be7320c34f3bab091c92ec2c1f0c]
I20260808 13:35:52.591547 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_bf5259e68f5c4869bac79b5b0a6a0cd0 [id=acbe4578625746948e76ae16ce884569]
I20260808 13:35:52.592810 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_834da6ca85d64417964c7976b57c1329 [id=ad4e3ed6750e4b7b8262ec862dd1c699]
I20260808 13:35:52.594034 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_280edf174ae84743aac082cfd2788315 [id=ae895d0078c748d78940cf2152e0e635]
I20260808 13:35:52.594017 8698 ts_tablet_manager.cc:1905] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:52.607455 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_233ac5cc38504a5eb9b9cbf426cc19c7 [id=af41016671a644dcaacf0b2602d83672]
I20260808 13:35:52.609560 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_da0f5516539d4f17b7ad03013cfd9460 [id=b06d9584a664410aad7628dbb2495f85]
I20260808 13:35:52.610914 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_c7297f7910b4405ab0483761e1aed004 [id=b0736fe025cb443ba8cbab989b9a4422]
I20260808 13:35:52.612234 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_83babd1cd9304044b4bdb5cf0ee909a4 [id=b1e1dc25e25a4075ae747315c8130723]
W20260808 13:35:52.494978 5283 leader_election.cc:341] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.442636 6046 consensus_queue.cc:260] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.579886 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d12106f0d794a79935999a2b8dbe7e9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:52.581977 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09eb729339e47eb8db88eb6509daab4" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:52.583142 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9e8d48c55349058cf80bf7c2e149e7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:52.621817 8684 ts_tablet_manager.cc:1918] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260808 13:35:52.633527 7535 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 58597us
I20260808 13:35:52.642241 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_95ed126fb41342c1a6a1539f0a5ebbe3 [id=b301b3d4964b41009ed2903dcbefdfae]
W20260808 13:35:52.668710 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 173752us
I20260808 13:35:52.675864 6046 ts_tablet_manager.cc:1428] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.484s user 0.007s sys 0.000s
I20260808 13:35:52.646351 6033 tablet_bootstrap.cc:492] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:52.680075 5285 leader_election.cc:302] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:52.682433 8706 raft_consensus.cc:2781] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:52.682930 8706 raft_consensus.cc:686] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:52.683677 8706 consensus_queue.cc:237] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.684441 6033 ts_tablet_manager.cc:1397] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.246s user 0.015s sys 0.003s
I20260808 13:35:52.672174 8684 log.cc:1196] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/851a0b374a854efd87dd66e39b9df2d5
I20260808 13:35:52.688423 8684 ts_tablet_manager.cc:1939] T 851a0b374a854efd87dd66e39b9df2d5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
W20260808 13:35:52.724634 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 231897us
I20260808 13:35:52.727571 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:52.733448 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 7e8cf06462484c99a860762a82f7ad64 (table default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]) successfully deleted
I20260808 13:35:52.720247 6033 raft_consensus.cc:348] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:52.749588 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 79278.7us
I20260808 13:35:52.646534 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_a1db036771e0467fb083f890fefd54e9 [id=b449f7fe2512431790a1a39a4dde3570]
I20260808 13:35:52.751822 6033 raft_consensus.cc:374] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:52.763520 6033 raft_consensus.cc:729] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:52.764417 6033 consensus_queue.cc:260] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.771160 6046 tablet_bootstrap.cc:492] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:52.775955 4820 leader_election.cc:334] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555): Timed out: RequestConsensusVote RPC to 127.4.170.61:46555 timed out after 0.728s (SENT)
I20260808 13:35:52.776469 4820 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:35:52.810107 8788 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:35:52.762369 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212]
I20260808 13:35:52.841693 6046 tablet_bootstrap.cc:654] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:52.853155 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 127394us
W20260808 13:35:52.867519 8784 log.cc:924] Time spent T 023fe27fc8ce4321a43a1f6037dd9cce P 946c9d8462644deba5b45fa724f1923c: Append to log took a long time: real 0.081s user 0.000s sys 0.000s
W20260808 13:35:52.870754 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 78461.6us
I20260808 13:35:52.880503 8743 raft_consensus.cc:482] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:52.881086 8743 raft_consensus.cc:504] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:52.883014 8743 leader_election.cc:288] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:52.897284 5551 leader_election.cc:341] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.897828 5551 leader_election.cc:302] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:52.898623 8792 raft_consensus.cc:2726] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:52.911558 8700 ts_tablet_manager.cc:1918] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:52.912045 8700 log.cc:1196] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/0756b5f24ed94a9fbe2a43efc7b8d40d
I20260808 13:35:52.912820 8700 ts_tablet_manager.cc:1939] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:52.915220 8700 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 474) took 1280 ms. Trace:
I20260808 13:35:52.915596 8700 rpcz_store.cc:276] 0808 13:35:51.634505 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.634635 (+ 130us) service_pool.cc:227] Handling call
0808 13:35:52.216426 (+581791us) ts_tablet_manager.cc:1145] Deleting tablet 0756b5f24ed94a9fbe2a43efc7b8d40d
0808 13:35:52.216464 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:52.856735 (+640271us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:52.911288 (+ 54553us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:52.915154 (+ 3866us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":579898}
I20260808 13:35:52.917207 8700 tablet_replica.cc:330] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:35:52.931059 6046 tablet_bootstrap.cc:492] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:52.931689 6046 ts_tablet_manager.cc:1397] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.161s user 0.011s sys 0.003s
I20260808 13:35:52.960528 8773 raft_consensus.cc:482] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:52.961225 8773 raft_consensus.cc:504] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:52.964437 8773 leader_election.cc:288] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:52.971057 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 73808.5us
W20260808 13:35:52.971805 5551 leader_election.cc:341] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:52.972270 5551 leader_election.cc:302] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:52.973073 8773 raft_consensus.cc:2726] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:52.979398 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_fbaf65bcf1fd48249596df8d79af088b [id=b529cc209a934207ab4940d51ee4876b]
I20260808 13:35:52.981913 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_0ab0a38241284050a12358d0f8d76032 [id=b713080d065a4fa49a583244a722909c]
I20260808 13:35:52.986070 6026 tablet_bootstrap.cc:654] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:52.956874 6033 ts_tablet_manager.cc:1428] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.271s user 0.014s sys 0.002s
I20260808 13:35:52.934576 6046 raft_consensus.cc:348] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.032141 6046 raft_consensus.cc:374] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:53.032475 6046 raft_consensus.cc:729] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:35:52.997140 5283 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.4.170.62:34069, user_credentials={real_user=slave}} blocked reactor thread for 120898us
I20260808 13:35:53.064224 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]
W20260808 13:35:53.023284 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 51501.9us
W20260808 13:35:53.071146 5551 leader_election.cc:341] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:53.071661 5551 leader_election.cc:302] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:53.060376 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d31516006bc4a65b496dcf363db464a" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:53.072856 5634 raft_consensus.cc:2379] T 7d31516006bc4a65b496dcf363db464a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:53.075100 8773 raft_consensus.cc:2726] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:53.076227 5551 leader_election.cc:341] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:53.033622 6046 consensus_queue.cc:260] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.087921 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet 0756b5f24ed94a9fbe2a43efc7b8d40d (table default.Table_a730d953de56401597f5e28a6edc707a [id=ba2a2d02f38d47a489d40b228fe0af55]) successfully deleted
I20260808 13:35:53.093328 5551 leader_election.cc:302] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:53.094534 8773 raft_consensus.cc:2726] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:53.096877 6046 ts_tablet_manager.cc:1428] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.165s user 0.002s sys 0.004s
I20260808 13:35:53.107528 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_e70822ac71fa4d0e9205367df025ad3a [id=b809e4f7fa074a608bceba5f738df6a6]
I20260808 13:35:53.113631 8700 raft_consensus.cc:2229] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:53.114226 8700 raft_consensus.cc:2258] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:53.116320 8700 ts_tablet_manager.cc:1905] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:53.128237 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba39706cff65404485959865fff1772a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:53.135625 6046 tablet_bootstrap.cc:492] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:53.147063 5285 consensus_peers.cc:487] T 0756b5f24ed94a9fbe2a43efc7b8d40d P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0756b5f24ed94a9fbe2a43efc7b8d40d. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:53.154529 8706 consensus_queue.cc:1035] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:53.157763 8698 ts_tablet_manager.cc:1918] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:53.158344 8698 log.cc:1196] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/851a0b374a854efd87dd66e39b9df2d5
I20260808 13:35:53.159358 8698 ts_tablet_manager.cc:1939] T 851a0b374a854efd87dd66e39b9df2d5 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:53.176311 8796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:35:53.196451 8796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:35:53.199498 6046 tablet_bootstrap.cc:654] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:53.218079 8807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:35:53.219010 8807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 13:35:53.220201 5551 leader_election.cc:341] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:53.185007 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_51d39ab358f3470084d7cd6e270eee0c [id=bbe62ee3dd5d494d935378ada5650363]
I20260808 13:35:53.228974 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_6c19755f3f9149e7bdac49809abbe2a3 [id=bd6ed2fb84f44308a249c7b2cfa8d6a3]
I20260808 13:35:53.239682 6026 tablet_bootstrap.cc:492] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:53.240372 6026 ts_tablet_manager.cc:1397] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.784s user 0.012s sys 0.005s
W20260808 13:35:53.254424 7535 leader_election.cc:334] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555): Timed out: RequestConsensusVote RPC to 127.4.170.61:46555 timed out after 0.618s (SENT)
I20260808 13:35:53.255129 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:53.256162 8807 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:35:53.256757 6026 raft_consensus.cc:348] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:53.216614 8698 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 492) took 1660 ms. Trace:
I20260808 13:35:53.262075 8698 rpcz_store.cc:276] 0808 13:35:51.556611 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.556708 (+ 97us) service_pool.cc:227] Handling call
0808 13:35:52.447295 (+890587us) ts_tablet_manager.cc:1145] Deleting tablet 851a0b374a854efd87dd66e39b9df2d5
0808 13:35:52.447348 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:52.720268 (+272920us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:53.157481 (+437213us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:53.216534 (+ 59053us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":890038}
I20260808 13:35:53.267458 6026 raft_consensus.cc:374] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:53.269237 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 851a0b374a854efd87dd66e39b9df2d5 (table default.Table_6ba381f21fc24362bc6d18499af1d329 [id=393e6ebfd240438297030391565be087]) successfully deleted
I20260808 13:35:53.271610 8698 tablet_replica.cc:330] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:53.282488 8700 ts_tablet_manager.cc:1918] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:53.282982 8700 log.cc:1196] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/ba39706cff65404485959865fff1772a
I20260808 13:35:53.283070 6026 raft_consensus.cc:729] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:53.293702 6026 consensus_queue.cc:260] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:53.307317 8698 raft_consensus.cc:2229] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:53.308154 8698 raft_consensus.cc:2258] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:53.315191 8700 ts_tablet_manager.cc:1939] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:53.322831 6026 ts_tablet_manager.cc:1428] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.082s user 0.007s sys 0.000s
I20260808 13:35:53.327872 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_b8623fa3d1eb479a96e59b16c329e590 [id=c0f00fb75b204087bd476376ddfdc046]
I20260808 13:35:53.378054 8613 raft_consensus.cc:482] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:53.379071 8613 raft_consensus.cc:504] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:53.390771 8613 leader_election.cc:288] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:53.395365 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_a450e64d7a494bbcb8f35780fda0ccc8 [id=c4338f1c19ec4210a803a1e2aaaa65d4]
I20260808 13:35:53.407307 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_6ea0a150e9a245babc374b7222ad1ca7 [id=c69587f42e4b437aadf52a706e37e070]
I20260808 13:35:53.424914 8684 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 475) took 2208 ms. Trace:
I20260808 13:35:53.425419 6033 tablet_bootstrap.cc:492] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:53.439841 8684 rpcz_store.cc:276] 0808 13:35:51.216029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.217098 (+ 1069us) service_pool.cc:227] Handling call
0808 13:35:52.271037 (+1053939us) ts_tablet_manager.cc:1145] Deleting tablet 851a0b374a854efd87dd66e39b9df2d5
0808 13:35:52.271080 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:52.574063 (+302983us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:52.621566 (+ 47503us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:53.424519 (+802953us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1053327}
I20260808 13:35:53.447387 8817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:35:53.448107 8817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:53.463528 6033 tablet_bootstrap.cc:654] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:53.468137 8814 raft_consensus.cc:482] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:53.469231 8814 raft_consensus.cc:504] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.475276 8814 leader_election.cc:288] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:53.493002 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 851a0b374a854efd87dd66e39b9df2d5 (table default.Table_6ba381f21fc24362bc6d18499af1d329 [id=393e6ebfd240438297030391565be087]) successfully deleted
I20260808 13:35:53.442951 8684 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 479) took 2204 ms. Trace:
I20260808 13:35:53.497777 8684 rpcz_store.cc:276] 0808 13:35:51.238680 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.238789 (+ 109us) service_pool.cc:227] Handling call
0808 13:35:53.442309 (+2203520us) ts_tablet_manager.cc:1145] Deleting tablet 369a942248bd4cec9ee9170fe55601a3
0808 13:35:53.442436 (+ 127us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:53.442872 (+ 436us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2202891}
I20260808 13:35:53.499446 8684 tablet_replica.cc:330] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
W20260808 13:35:53.503832 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 369a942248bd4cec9ee9170fe55601a3 because the tablet was not found. No further retry: Not found: Tablet not found: 369a942248bd4cec9ee9170fe55601a3
I20260808 13:35:53.525918 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba39706cff65404485959865fff1772a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:53.531193 8684 raft_consensus.cc:2229] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:35:53.535946 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_efea7e3ddc70498f94c05ddee5ba3d76 [id=c965816b949348ba98967bdc0115eefa]
I20260808 13:35:53.537854 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88]
I20260808 13:35:53.538944 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b314734698ed423ba9cf4134db5846ec" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:53.540462 8817 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:35:53.541524 8814 raft_consensus.cc:482] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 946c9d8462644deba5b45fa724f1923c)
I20260808 13:35:53.544312 7736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:35:53.547854 8652 consensus_queue.cc:1035] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:53.542297 8814 raft_consensus.cc:504] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:53.550276 8814 leader_election.cc:288] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:53.560240 5550 leader_election.cc:341] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:53.560707 5550 leader_election.cc:302] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:53.561319 5550 raft_consensus.cc:2670] T ba39706cff65404485959865fff1772a P 5cd987c3dbdb4168beb73cd7e577c7cb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260808 13:35:53.562335 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7454bf27bc1a45d2af0d8f897b2f9f97" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:53.563051 5638 raft_consensus.cc:2454] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:53.572993 6026 tablet_bootstrap.cc:492] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:53.574055 8700 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 475) took 1936 ms. Trace:
I20260808 13:35:53.574431 8700 rpcz_store.cc:276] 0808 13:35:51.637733 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.637824 (+ 91us) service_pool.cc:227] Handling call
0808 13:35:52.916901 (+1279077us) ts_tablet_manager.cc:1145] Deleting tablet ba39706cff65404485959865fff1772a
0808 13:35:52.916983 (+ 82us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:53.156724 (+239741us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:53.282222 (+125498us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:53.573986 (+291764us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":195231,"tablet-delete.queue_time_us":1278503}
I20260808 13:35:53.575889 8700 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 476) took 1874 ms. Trace:
I20260808 13:35:53.576210 8700 rpcz_store.cc:276] 0808 13:35:51.701074 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.701178 (+ 104us) service_pool.cc:227] Handling call
0808 13:35:53.575583 (+1874405us) ts_tablet_manager.cc:1145] Deleting tablet ba39706cff65404485959865fff1772a
0808 13:35:53.575624 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:53.575834 (+ 210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1873856}
I20260808 13:35:53.577337 8700 tablet_replica.cc:330] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
W20260808 13:35:53.581606 5550 leader_election.cc:341] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:53.582103 6026 tablet_bootstrap.cc:654] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:53.554126 6033 tablet_bootstrap.cc:492] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:53.592931 8684 raft_consensus.cc:2258] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:53.602629 6033 ts_tablet_manager.cc:1397] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.178s user 0.009s sys 0.007s
I20260808 13:35:53.605727 6033 raft_consensus.cc:348] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.606519 6033 raft_consensus.cc:374] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:53.606850 6033 raft_consensus.cc:729] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:53.607510 6033 consensus_queue.cc:260] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.610657 6033 ts_tablet_manager.cc:1428] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260808 13:35:53.627562 6026 tablet_bootstrap.cc:492] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:53.629323 6026 ts_tablet_manager.cc:1397] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.057s user 0.012s sys 0.000s
I20260808 13:35:53.633052 8700 raft_consensus.cc:2229] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:53.633807 8700 raft_consensus.cc:2258] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:53.636044 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_f676fb75c0ba45a98625440ed5dd7313 [id=d342034fcd7b45debb1aca898c6b7f68]
I20260808 13:35:53.636111 8700 ts_tablet_manager.cc:1905] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:53.646142 6026 raft_consensus.cc:348] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.655224 6026 raft_consensus.cc:374] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:53.656076 6026 raft_consensus.cc:729] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:53.657425 6026 consensus_queue.cc:260] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.669600 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_7a1963a007954f8ca1d330f9d1936dd3 [id=d5b4528cac46486e857db6176b4b3a2b]
I20260808 13:35:53.679633 6026 ts_tablet_manager.cc:1428] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.046s user 0.004s sys 0.002s
I20260808 13:35:53.682387 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_f1ad878382f04d038b6b08429d6b9db9 [id=d84314027def49d380b8c13a0431f39e]
I20260808 13:35:53.690315 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc]
I20260808 13:35:53.704780 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_953730643273433c8353604101a4d237 [id=d9baa62ea5c249a898586a45f8b4f372]
I20260808 13:35:53.707161 6026 tablet_bootstrap.cc:492] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:53.713243 6026 tablet_bootstrap.cc:654] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:53.721762 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_e101c061f2bf4821b6fe2c6108432735 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7]
I20260808 13:35:53.728147 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_82d49330ca914820a6cb8b3591280a90 [id=dc1f8a5aa37a4ad99be0715831a680fd]
I20260808 13:35:53.728425 8698 ts_tablet_manager.cc:1905] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:53.731700 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet ba39706cff65404485959865fff1772a (table default.Table_fc7cb305e35541c781c3e3790cf8a0af [id=b885c5ab382c45809d6f896ebf8771bd]) successfully deleted
W20260808 13:35:53.733603 4821 catalog_manager.cc:4808] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet ba39706cff65404485959865fff1772a because the tablet was not found. No further retry: Not found: Tablet not found: ba39706cff65404485959865fff1772a
I20260808 13:35:53.743304 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d31516006bc4a65b496dcf363db464a" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:53.744091 6033 tablet_bootstrap.cc:492] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:53.758013 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_f998dcf284c249d7a6db0e4b803a157a [id=dd464b6ca5eb49499b0921fa2b99cb29]
I20260808 13:35:53.768081 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_5db9de8011f74b9f9192b39113dfa0c8 [id=dfd09716123a4de7b9fcdd65d1b9ef61]
I20260808 13:35:53.768281 6026 tablet_bootstrap.cc:492] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:53.768963 6026 ts_tablet_manager.cc:1397] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.062s user 0.001s sys 0.011s
I20260808 13:35:53.773967 6026 raft_consensus.cc:348] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:53.775228 6026 raft_consensus.cc:374] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:53.775569 6026 raft_consensus.cc:729] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:53.776367 6026 consensus_queue.cc:260] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:53.779580 6026 ts_tablet_manager.cc:1428] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20260808 13:35:53.793982 6026 tablet_bootstrap.cc:492] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:53.807739 8684 ts_tablet_manager.cc:1905] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:53.812414 6033 tablet_bootstrap.cc:654] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:53.825735 8700 ts_tablet_manager.cc:1918] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:35:53.862957 8700 log.cc:1196] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/de6800eceb96475284fdbffb4d23e5e3
I20260808 13:35:53.863869 8700 ts_tablet_manager.cc:1939] T de6800eceb96475284fdbffb4d23e5e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:35:53.769695 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_0913077ace074f11bf8a6222338489c6 [id=e048189f6ef441ce8e17c60f0f5b0cd7]
I20260808 13:35:53.912257 8700 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50600 (request call id 477) took 2210 ms. Trace:
I20260808 13:35:53.912949 8700 rpcz_store.cc:276] 0808 13:35:51.702247 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.702343 (+ 96us) service_pool.cc:227] Handling call
0808 13:35:53.577168 (+1874825us) ts_tablet_manager.cc:1145] Deleting tablet de6800eceb96475284fdbffb4d23e5e3
0808 13:35:53.577211 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:53.646269 (+ 69058us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:53.825409 (+179140us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:53.912181 (+ 86772us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":54483,"spinlock_wait_cycles":2816,"tablet-delete.queue_time_us":1874281}
I20260808 13:35:53.914857 4821 catalog_manager.cc:4841] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): tablet de6800eceb96475284fdbffb4d23e5e3 (table default.Table_9fa5c49027684fe6a85a46895e121932 [id=7fff2277bc2f4edd9aeccc15d58333c3]) successfully deleted
I20260808 13:35:53.898315 5772 leader_election.cc:302] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:53.920917 8654 raft_consensus.cc:2781] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:53.921258 8654 raft_consensus.cc:482] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:53.921545 8654 raft_consensus.cc:3037] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:53.923331 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 8d17615af06b4144a0d7d74c64416451 (table default.Table_f10f80e85d6c4f87b43c1a53b888716c [id=20c27e5e63df49a8bcf6faef31940880], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8d17615af06b4144a0d7d74c64416451 consensus state has no leader
I20260808 13:35:53.927956 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_f985e4cebca740fea14f259fcfacec86 [id=e0c4baae657c406b9d7c8c59fb77e10e]
I20260808 13:35:53.929719 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_41f3478fe8c94fbfaf7fcbdee22ce987 [id=e3e121c36ff44ea0a3a98185c8cce381]
I20260808 13:35:53.931499 8654 raft_consensus.cc:504] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.933027 8773 raft_consensus.cc:482] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:53.933538 8654 leader_election.cc:288] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:53.933594 8773 raft_consensus.cc:504] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.936547 8773 leader_election.cc:288] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:53.937299 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c41483119a442eb46816aa34f7f319" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:53.943475 6046 tablet_bootstrap.cc:492] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
W20260808 13:35:53.946741 5550 leader_election.cc:341] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:53.955070 5376 raft_consensus.cc:3037] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:53.959472 5383 raft_consensus.cc:3037] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:53.975524 6046 ts_tablet_manager.cc:1397] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.856s user 0.013s sys 0.000s
I20260808 13:35:53.978111 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7454bf27bc1a45d2af0d8f897b2f9f97" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:53.989328 8837 consensus_queue.cc:1035] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:53.992987 6046 raft_consensus.cc:348] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.993922 6046 raft_consensus.cc:374] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:53.994148 6026 tablet_bootstrap.cc:654] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:53.994231 6046 raft_consensus.cc:729] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:53.994833 6046 consensus_queue.cc:260] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:53.996982 6046 ts_tablet_manager.cc:1428] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.021s user 0.000s sys 0.006s
I20260808 13:35:54.002893 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_7700dbef158a4ddfa8a3122c18e38817 [id=e6381ed760224b53be85a88376ce8962]
I20260808 13:35:54.027799 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7454bf27bc1a45d2af0d8f897b2f9f97" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:54.036095 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b314734698ed423ba9cf4134db5846ec" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:54.055110 8743 raft_consensus.cc:482] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:54.055747 8743 raft_consensus.cc:504] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.060415 8743 leader_election.cc:288] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:54.060807 6026 tablet_bootstrap.cc:492] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:54.061030 8837 raft_consensus.cc:482] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 98dd96f425fc4ab3a6a54f2dfaaf9fab)
I20260808 13:35:54.062187 6046 tablet_bootstrap.cc:492] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:54.061725 8837 raft_consensus.cc:504] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.069929 6033 tablet_bootstrap.cc:492] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:54.070361 6033 ts_tablet_manager.cc:1397] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.327s user 0.010s sys 0.011s
I20260808 13:35:54.072999 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_533328cbcc13486fb2b3c66451f55ac3 [id=e90cdc01582749b092c452f915c314eb]
I20260808 13:35:54.074018 6026 ts_tablet_manager.cc:1397] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.280s user 0.012s sys 0.000s
I20260808 13:35:54.064993 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:54.081676 5636 raft_consensus.cc:2454] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:54.082190 8853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:35:54.089242 8853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:54.091392 8853 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:54.092362 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:54.087216 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_f042821be4a8421ea3f64f33a071b22d [id=ea2ba343c17d403d8cb0bf9d80bee453]
I20260808 13:35:54.110481 6026 raft_consensus.cc:348] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.120128 8654 raft_consensus.cc:482] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:54.120735 6026 raft_consensus.cc:374] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:54.120637 8654 raft_consensus.cc:504] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.121559 6026 raft_consensus.cc:729] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:54.122521 6026 consensus_queue.cc:260] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.126560 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_9d2535df26964377be89e6da0a54e8b4 [id=eb356c7ce7ca4cc1a5a51234f3404e3c]
W20260808 13:35:54.061685 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 163827us
W20260808 13:35:54.137235 5772 leader_election.cc:341] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 4ff163c36c6d42238cce2396102f9a68
I20260808 13:35:54.137530 8821 consensus_queue.cc:1035] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:54.138031 8654 leader_election.cc:288] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:54.147853 8837 leader_election.cc:288] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:54.149466 5771 consensus_peers.cc:487] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e8cf06462484c99a860762a82f7ad64. This is attempt 1: this message will repeat every 5th retry.
W20260808 13:35:54.150863 5771 leader_election.cc:341] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:54.152282 5771 consensus_peers.cc:487] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:54.156137 6026 ts_tablet_manager.cc:1428] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.082s user 0.008s sys 0.000s
I20260808 13:35:54.156028 8684 ts_tablet_manager.cc:1918] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:54.156559 8684 log.cc:1196] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/362dbd6990e740b39690537af98aa2f8
I20260808 13:35:54.156698 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c41483119a442eb46816aa34f7f319" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:54.157573 8684 ts_tablet_manager.cc:1939] T 362dbd6990e740b39690537af98aa2f8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:54.158546 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_45fdb943ae1b499e98db51a5ee6589a2 [id=ed0521e8ecae4c609b2a7426e5a565c0]
W20260808 13:35:54.159204 5771 leader_election.cc:341] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Not found: Tablet not found: 4ff163c36c6d42238cce2396102f9a68
I20260808 13:35:54.159960 5771 leader_election.cc:302] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:54.160332 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_52f5a796f61f4be79a196bc23065491b [id=ee175d53df794e6aaf29faff1fc2bb6f]
I20260808 13:35:54.161003 8821 raft_consensus.cc:2726] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:54.163228 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_cab0814763554410aa969d5b417710fb [id=eef293a0bbc94c2db94a64223ef71a5d]
I20260808 13:35:54.164649 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_c171ae1c3b234b4e8b6f3def21f27924 [id=efc0b7748878445c8245bcd3e9657abe]
W20260808 13:35:54.165236 5771 leader_election.cc:341] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.165901 5285 leader_election.cc:302] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:54.166734 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_e916116a3ec24469b4670c87eeaac98e [id=f348502b786943fdb6bcb104e73355f4]
W20260808 13:35:54.166842 5285 leader_election.cc:341] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 86c2b34c95244472beec43643e60ff5f
I20260808 13:35:54.167698 8837 raft_consensus.cc:2781] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:35:54.168105 8837 raft_consensus.cc:482] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:54.168470 8837 raft_consensus.cc:3037] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:35:54.169366 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_ddfcf011ede6442fbc7a45c799fc2cc6 [id=f3bc7b846abd4480b1c1c927c8ad0953]
I20260808 13:35:54.170701 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_5f0b33d3ad8942c0adb8ded9e15220c3 [id=f432910006cd459b9c1c110e9a4a3bda]
W20260808 13:35:54.171716 5551 leader_election.cc:341] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.172139 5551 leader_election.cc:302] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:54.172412 8821 consensus_queue.cc:1035] T 7d31516006bc4a65b496dcf363db464a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:54.172806 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e36d643850411498c23699835f55a6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:54.174234 8773 raft_consensus.cc:2726] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:54.174566 6026 tablet_bootstrap.cc:492] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:54.174533 8821 consensus_queue.cc:1035] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:54.186336 8613 raft_consensus.cc:482] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:54.186892 8613 raft_consensus.cc:504] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.188939 8613 leader_election.cc:288] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:54.189248 6046 tablet_bootstrap.cc:654] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:54.193449 6026 tablet_bootstrap.cc:654] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:54.148825 8698 ts_tablet_manager.cc:1918] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:54.204056 8698 log.cc:1196] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/362dbd6990e740b39690537af98aa2f8
I20260808 13:35:54.205189 8698 ts_tablet_manager.cc:1939] T 362dbd6990e740b39690537af98aa2f8 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:54.146409 6033 raft_consensus.cc:348] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.213529 6033 raft_consensus.cc:374] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:54.213785 6033 raft_consensus.cc:729] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:54.214367 6033 consensus_queue.cc:260] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.218282 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_0a849c19427549aea0eabc1e0324725f [id=f58086e90fc34ca3a5d31417a82dad6c]
I20260808 13:35:54.219199 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01ca2e4d6a2a4e2bb2dd63af8c125e9f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:54.221328 5550 leader_election.cc:341] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:54.210708 5283 consensus_peers.cc:487] T 1ba5fa465ec04bbb801a821a71a8f87d P 946c9d8462644deba5b45fa724f1923c -> Peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Couldn't send request to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:54.234535 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_d39fd423849c45c69b7338e8c6a87c30 [id=f5bbd5d961794ddcab1f07f3931ab031]
W20260808 13:35:54.238982 4820 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 45411.1us
W20260808 13:35:54.238978 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 65580.2us
W20260808 13:35:54.245596 5551 leader_election.cc:341] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.246138 5551 leader_election.cc:302] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:54.247000 8773 raft_consensus.cc:2726] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:54.255883 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_465ac7f2a60c415a8809fdfccb4b5e27 [id=f5be929e60354ded904b228314f820d9]
I20260808 13:35:54.258442 8744 consensus_queue.cc:1035] T eaa049ef916b44458be43ec4e2ccaf1c P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260808 13:35:54.261046 6026 tablet_bootstrap.cc:492] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:54.261459 6026 ts_tablet_manager.cc:1397] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.087s user 0.009s sys 0.000s
I20260808 13:35:54.261607 8684 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 480) took 3021 ms. Trace:
I20260808 13:35:54.262071 8684 rpcz_store.cc:276] 0808 13:35:51.239711 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.239842 (+ 131us) service_pool.cc:227] Handling call
0808 13:35:53.499192 (+2259350us) ts_tablet_manager.cc:1145] Deleting tablet 362dbd6990e740b39690537af98aa2f8
0808 13:35:53.499255 (+ 63us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:53.868137 (+368882us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:54.155750 (+287613us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:54.261540 (+105790us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":92621,"tablet-delete.queue_time_us":2258848}
I20260808 13:35:54.265718 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 362dbd6990e740b39690537af98aa2f8 (table default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]) successfully deleted
I20260808 13:35:54.257294 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_660efdc735cf4c31bf966be22b38dc28 [id=f603e442844e4a09bfecb79b93014265]
I20260808 13:35:54.268949 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_de3d9a6e588d44f698c656a21558e2ae [id=f92accd458dd439abf6c75dd1cd79df5]
I20260808 13:35:54.271564 6026 raft_consensus.cc:348] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.273813 5283 leader_election.cc:302] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:54.277093 6026 raft_consensus.cc:374] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:54.277443 6026 raft_consensus.cc:729] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:54.279667 8698 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 493) took 2722 ms. Trace:
W20260808 13:35:54.283057 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 144656us
I20260808 13:35:54.281773 6026 consensus_queue.cc:260] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.285933 6026 ts_tablet_manager.cc:1428] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.024s user 0.007s sys 0.000s
I20260808 13:35:54.286839 6026 tablet_bootstrap.cc:492] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:54.287403 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_6b4561dcd8e64216a27618e3599c13d8 [id=fce834aa5aec4694845b2ffc2c9fe962]
I20260808 13:35:54.290946 6033 ts_tablet_manager.cc:1428] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.220s user 0.009s sys 0.000s
I20260808 13:35:54.304390 8821 raft_consensus.cc:482] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:54.304864 8821 raft_consensus.cc:504] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.307147 8821 leader_election.cc:288] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:54.316893 8684 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 478) took 3081 ms. Trace:
I20260808 13:35:54.317368 8684 rpcz_store.cc:276] 0808 13:35:51.235038 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.235206 (+ 168us) service_pool.cc:227] Handling call
0808 13:35:54.266815 (+3031609us) ts_tablet_manager.cc:1145] Deleting tablet 196d80d806b0432cacf2ca6aa0da8c6b
0808 13:35:54.314677 (+ 47862us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:54.316752 (+ 2075us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3025984}
I20260808 13:35:54.281065 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01ca2e4d6a2a4e2bb2dd63af8c125e9f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:54.319031 4819 catalog_manager.cc:4808] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 196d80d806b0432cacf2ca6aa0da8c6b because the tablet was not found. No further retry: Not found: Tablet not found: 196d80d806b0432cacf2ca6aa0da8c6b
I20260808 13:35:54.320386 8684 tablet_replica.cc:330] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:54.319065 8698 rpcz_store.cc:276] 0808 13:35:51.557026 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.557454 (+ 428us) service_pool.cc:227] Handling call
0808 13:35:53.271333 (+1713879us) ts_tablet_manager.cc:1145] Deleting tablet 362dbd6990e740b39690537af98aa2f8
0808 13:35:53.271418 (+ 85us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:53.972889 (+701471us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:54.148548 (+175659us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:54.279569 (+131021us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1713282}
I20260808 13:35:54.339608 6033 tablet_bootstrap.cc:492] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:54.339673 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_ec97d7d5adc245a6b9d4224e8ab75cd5 [id=fd9c3138231f42b791685b0df92ac968]
I20260808 13:35:54.292196 8858 raft_consensus.cc:482] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:54.343205 8858 raft_consensus.cc:504] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.345227 8858 leader_election.cc:288] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:54.299825 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7454bf27bc1a45d2af0d8f897b2f9f97" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:54.345847 5636 raft_consensus.cc:3037] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:54.345950 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356e15bbcd26409a9d6302599816e15f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:54.301456 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e36d643850411498c23699835f55a6" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:54.348505 5550 leader_election.cc:341] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.283293 8837 raft_consensus.cc:504] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.305110 8744 raft_consensus.cc:2726] T 7d31516006bc4a65b496dcf363db464a P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:54.355477 5377 raft_consensus.cc:1226] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Rejecting Update request from peer 98dd96f425fc4ab3a6a54f2dfaaf9fab for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260808 13:35:54.356324 8837 leader_election.cc:288] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:54.357412 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:54.360833 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6800eceb96475284fdbffb4d23e5e3" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:54.361541 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 362dbd6990e740b39690537af98aa2f8 (table default.Table_203fa9f53a2345519f4a5825d2661b97 [id=7b1287cda59f41cc88f8ed19c56ac1c2]) successfully deleted
I20260808 13:35:54.354729 8567 catalog_manager.cc:669] Loaded metadata for table default.Table_93429697aa7c4405b439864c62c45c29 [id=feca1558e3e74fb2a9de982f5a2a5302]
I20260808 13:35:54.369509 5636 raft_consensus.cc:2454] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
W20260808 13:35:54.370793 5551 leader_election.cc:341] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.376905 5551 leader_election.cc:302] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:54.377761 8858 raft_consensus.cc:2726] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:54.380086 6026 tablet_bootstrap.cc:654] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:54.322048 8698 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 491) took 2765 ms. Trace:
I20260808 13:35:54.380733 8698 rpcz_store.cc:276] 0808 13:35:51.556343 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.556460 (+ 117us) service_pool.cc:227] Handling call
0808 13:35:54.321417 (+2764957us) ts_tablet_manager.cc:1145] Deleting tablet daa59a18ae624bc8975bee2c460895a7
0808 13:35:54.321553 (+ 136us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:54.321961 (+ 408us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2721964}
W20260808 13:35:54.385628 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet daa59a18ae624bc8975bee2c460895a7 because the tablet was not found. No further retry: Not found: Tablet not found: daa59a18ae624bc8975bee2c460895a7
I20260808 13:35:54.395282 8684 raft_consensus.cc:2229] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 LEADER]: Raft consensus shutting down.
I20260808 13:35:54.410825 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356e15bbcd26409a9d6302599816e15f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:54.412353 8815 consensus_queue.cc:1035] T 96bed5e6d1aa49a9b14ba86403c985ce P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:35:54.415352 5771 leader_election.cc:341] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.417024 8814 consensus_queue.cc:1046] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:54.417891 8814 raft_consensus.cc:3032] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Stepping down as leader of term 1
I20260808 13:35:54.418190 8814 raft_consensus.cc:729] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:54.418663 8698 tablet_replica.cc:330] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:35:54.419716 8698 raft_consensus.cc:2229] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 2 LEADER]: Raft consensus shutting down.
I20260808 13:35:54.420670 8814 consensus_queue.cc:260] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.421998 8814 raft_consensus.cc:3037] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Advancing to term 2
W20260808 13:35:54.424180 5551 leader_election.cc:341] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.424659 5551 leader_election.cc:302] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:54.425874 8793 raft_consensus.cc:2726] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:54.430521 6046 tablet_bootstrap.cc:492] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:54.431059 6046 ts_tablet_manager.cc:1397] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.369s user 0.012s sys 0.000s
I20260808 13:35:54.432466 6026 tablet_bootstrap.cc:492] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:54.433018 6026 ts_tablet_manager.cc:1397] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.146s user 0.010s sys 0.001s
I20260808 13:35:54.444752 5382 raft_consensus.cc:3037] T 96bed5e6d1aa49a9b14ba86403c985ce P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:54.447060 6026 raft_consensus.cc:348] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.449883 6026 raft_consensus.cc:374] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:54.450256 6026 raft_consensus.cc:729] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:54.458417 6046 raft_consensus.cc:348] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.455706 6026 consensus_queue.cc:260] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.459180 6046 raft_consensus.cc:374] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:54.461023 6026 ts_tablet_manager.cc:1428] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.028s user 0.003s sys 0.002s
I20260808 13:35:54.464968 8698 raft_consensus.cc:2258] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:54.466053 6026 tablet_bootstrap.cc:492] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:54.467120 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 135696us
I20260808 13:35:54.467332 8698 ts_tablet_manager.cc:1905] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:54.470530 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:54.472127 5869 raft_consensus.cc:3037] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:54.475540 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:54.477051 8684 raft_consensus.cc:2258] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:54.480798 8684 ts_tablet_manager.cc:1905] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:54.481950 8793 raft_consensus.cc:482] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:54.482388 8793 raft_consensus.cc:504] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.484489 8793 leader_election.cc:288] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:54.496596 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70a5cf88d94b36913a6d1ccfcb979a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:54.497939 5550 leader_election.cc:341] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.500622 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70a5cf88d94b36913a6d1ccfcb979a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:54.459724 6046 raft_consensus.cc:729] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:46.988135 7741 rpcz_store.cc:276] 0808 13:35:39.154033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:39.158633 (+ 4600us) service_pool.cc:227] Handling call
0808 13:35:40.269790 (+1111157us) raft_consensus.cc:1457] Updating replica for 483 ops
0808 13:35:40.422533 (+152743us) raft_consensus.cc:1520] Early marking committed up to index 0
0808 13:35:40.422578 (+ 45us) raft_consensus.cc:1525] Triggering prepare for 483 ops
0808 13:35:40.423235 (+ 657us) write_op.cc:270] Start()
0808 13:35:40.423398 (+ 163us) write_op.cc:276] Timestamp: P: 1786196103570066 usec, L: 0
0808 13:35:40.466924 (+ 43526us) write_op.cc:270] Start()
0808 13:35:40.467049 (+ 125us) write_op.cc:276] Timestamp: P: 1786196103574375 usec, L: 0
0808 13:35:40.467670 (+ 621us) write_op.cc:270] Start()
0808 13:35:40.467777 (+ 107us) write_op.cc:276] Timestamp: P: 1786196103576586 usec, L: 0
0808 13:35:40.468238 (+ 461us) write_op.cc:270] Start()
0808 13:35:40.468328 (+ 90us) write_op.cc:276] Timestamp: P: 1786196103579829 usec, L: 0
0808 13:35:40.468754 (+ 426us) write_op.cc:270] Start()
0808 13:35:40.468824 (+ 70us) write_op.cc:276] Timestamp: P: 1786196103585138 usec, L: 0
0808 13:35:40.469229 (+ 405us) write_op.cc:270] Start()
0808 13:35:40.469323 (+ 94us) write_op.cc:276] Timestamp: P: 1786196103647852 usec, L: 0
0808 13:35:40.471017 (+ 1694us) write_op.cc:270] Start()
0808 13:35:40.471174 (+ 157us) write_op.cc:276] Timestamp: P: 1786196103659888 usec, L: 0
0808 13:35:40.471668 (+ 494us) write_op.cc:270] Start()
0808 13:35:40.471747 (+ 79us) write_op.cc:276] Timestamp: P: 1786196103665177 usec, L: 0
0808 13:35:40.472217 (+ 470us) write_op.cc:270] Start()
0808 13:35:40.472315 (+ 98us) write_op.cc:276] Timestamp: P: 1786196103735291 usec, L: 0
0808 13:35:40.480982 (+ 8667us) write_op.cc:270] Start()
0808 13:35:40.481069 (+ 87us) write_op.cc:276] Timestamp: P: 1786196103764417 usec, L: 0
0808 13:35:40.481647 (+ 578us) write_op.cc:270] Start()
0808 13:35:40.481724 (+ 77us) write_op.cc:276] Timestamp: P: 1786196103899129 usec, L: 0
0808 13:35:40.482185 (+ 461us) write_op.cc:270] Start()
0808 13:35:40.482305 (+ 120us) write_op.cc:276] Timestamp: P: 1786196103931629 usec, L: 0
0808 13:35:40.482908 (+ 603us) write_op.cc:270] Start()
0808 13:35:40.482994 (+ 86us) write_op.cc:276] Timestamp: P: 1786196103986914 usec, L: 0
0808 13:35:40.483431 (+ 437us) write_op.cc:270] Start()
0808 13:35:40.483509 (+ 78us) write_op.cc:276] Timestamp: P: 1786196104031833 usec, L: 0
0808 13:35:40.485319 (+ 1810us) write_op.cc:270] Start()
0808 13:35:40.485402 (+ 83us) write_op.cc:276] Timestamp: P: 1786196104034132 usec, L: 0
0808 13:35:40.485803 (+ 401us) write_op.cc:270] Start()
0808 13:35:40.485874 (+ 71us) write_op.cc:276] Timestamp: P: 1786196104036665 usec, L: 0
0808 13:35:40.486312 (+ 438us) write_op.cc:270] Start()
0808 13:35:40.486380 (+ 68us) write_op.cc:276] Timestamp: P: 1786196104060100 usec, L: 0
0808 13:35:40.495585 (+ 9205us) write_op.cc:270] Start()
0808 13:35:40.495685 (+ 100us) write_op.cc:276] Timestamp: P: 1786196104100913 usec, L: 0
0808 13:35:40.496281 (+ 596us) write_op.cc:270] Start()
0808 13:35:40.496433 (+ 152us) write_op.cc:276] Timestamp: P: 1786196104200003 usec, L: 0
0808 13:35:40.496940 (+ 507us) write_op.cc:270] Start()
0808 13:35:40.497018 (+ 78us) write_op.cc:276] Timestamp: P: 1786196104206887 usec, L: 0
0808 13:35:40.497621 (+ 603us) write_op.cc:270] Start()
0808 13:35:40.497706 (+ 85us) write_op.cc:276] Timestamp: P: 1786196104238941 usec, L: 0
0808 13:35:40.498174 (+ 468us) write_op.cc:270] Start()
0808 13:35:40.498265 (+ 91us) write_op.cc:276] Timestamp: P: 1786196104274336 usec, L: 0
0808 13:35:40.498747 (+ 482us) write_op.cc:270] Start()
0808 13:35:40.498834 (+ 87us) write_op.cc:276] Timestamp: P: 1786196104406834 usec, L: 0
0808 13:35:40.499280 (+ 446us) write_op.cc:270] Start()
0808 13:35:40.499393 (+ 113us) write_op.cc:276] Timestamp: P: 1786196104409504 usec, L: 0
0808 13:35:40.499964 (+ 571us) write_op.cc:270] Start()
0808 13:35:40.500057 (+ 93us) write_op.cc:276] Timestamp: P: 1786196104489058 usec, L: 0
0808 13:35:40.500518 (+ 461us) write_op.cc:270] Start()
0808 13:35:40.501775 (+ 1257us) write_op.cc:276] Timestamp: P: 1786196104527570 usec, L: 0
0808 13:35:40.502385 (+ 610us) write_op.cc:270] Start()
0808 13:35:40.502472 (+ 87us) write_op.cc:276] Timestamp: P: 1786196104552155 usec, L: 0
0808 13:35:40.504126 (+ 1654us) write_op.cc:270] Start()
0808 13:35:40.504204 (+ 78us) write_op.cc:276] Timestamp: P: 1786196104622034 usec, L: 0
0808 13:35:40.504649 (+ 445us) write_op.cc:270] Start()
0808 13:35:40.504728 (+ 79us) write_op.cc:276] Timestamp: P: 1786196104731671 usec, L: 0
0808 13:35:40.505144 (+ 416us) write_op.cc:270] Start()
0808 13:35:40.505216 (+ 72us) write_op.cc:276] Timestamp: P: 1786196104763128 usec, L: 0
0808 13:35:40.505664 (+ 448us) write_op.cc:270] Start()
0808 13:35:40.505740 (+ 76us) write_op.cc:276] Timestamp: P: 1786196104765496 usec, L: 0
0808 13:35:40.506174 (+ 434us) write_op.cc:270] Start()
0808 13:35:40.506265 (+ 91us) write_op.cc:276] Timestamp: P: 1786196104770230 usec, L: 0
0808 13:35:40.506776 (+ 511us) write_op.cc:270] Start()
0808 13:35:40.506853 (+ 77us) write_op.cc:276] Timestamp: P: 1786196104805158 usec, L: 0
0808 13:35:40.507290 (+ 437us) write_op.cc:270] Start()
0808 13:35:40.507367 (+ 77us) write_op.cc:276] Timestamp: P: 1786196104861912 usec, L: 0
0808 13:35:40.530197 (+ 22830us) write_op.cc:270] Start()
0808 13:35:40.530309 (+ 112us) write_op.cc:276] Timestamp: P: 1786196104866242 usec, L: 0
0808 13:35:40.530819 (+ 510us) write_op.cc:270] Start()
0808 13:35:40.530897 (+ 78us) write_op.cc:276] Timestamp: P: 1786196104879499 usec, L: 0
0808 13:35:40.534117 (+ 3220us) write_op.cc:270] Start()
0808 13:35:40.534207 (+ 90us) write_op.cc:276] Timestamp: P: 1786196105020117 usec, L: 0
0808 13:35:40.534719 (+ 512us) write_op.cc:270] Start()
0808 13:35:40.534823 (+ 104us) write_op.cc:276] Timestamp: P: 1786196105029831 usec, L: 0
0808 13:35:40.535282 (+ 459us) write_op.cc:270] Start()
0808 13:35:40.535357 (+ 75us) write_op.cc:276] Timestamp: P: 1786196105041501 usec, L: 0
0808 13:35:40.535772 (+ 415us) write_op.cc:270] Start()
0808 13:35:40.535850 (+ 78us) write_op.cc:276] Timestamp: P: 1786196105097708 usec, L: 0
0808 13:35:40.536269 (+ 419us) write_op.cc:270] Start()
0808 13:35:40.536352 (+ 83us) write_op.cc:276] Timestamp: P: 1786196105208273 usec, L: 0
0808 13:35:40.536798 (+ 446us) write_op.cc:270] Start()
0808 13:35:40.536866 (+ 68us) write_op.cc:276] Timestamp: P: 1786196105390916 usec, L: 0
0808 13:35:40.567137 (+ 30271us) spinlock_profiling.cc:243] Waited 8.44 ms on lock 0x7b5400030300. stack: 00007ff0fbb8b033 00007ff0fbb8af30 00007ff0fb70c534 00000000004db2b4 00000000004db249 00007ff102ea7361 00007ff102f77a86 00007ff102f6de03 00007ff102f34f96 00007ff102f34acf 00007ff102b09040 00007ff102b12536 00007ff102b1199e 00007ff103aee2e9 00007ff1022c4684 00007ff1022c4612
0808 13:35:40.567781 (+ 644us) write_op.cc:270] Start()
0808 13:35:40.567876 (+ 95us) write_op.cc:276] Timestamp: P: 1786196105546702 usec, L: 0
0808 13:35:40.568459 (+ 583us) write_op.cc:270] Start()
0808 13:35:40.568583 (+ 124us) write_op.cc:276] Timestamp: P: 1786196105594807 usec, L: 0
0808 13:35:40.569145 (+ 562us) write_op.cc:270] Start()
0808 13:35:40.569232 (+ 87us) write_op.cc:276] Timestamp: P: 1786196105620418 usec, L: 0
0808 13:35:40.569794 (+ 562us) write_op.cc:270] Start()
0808 13:35:40.569874 (+ 80us) write_op.cc:276] Timestamp: P: 1786196105666469 usec, L: 0
0808 13:35:40.570299 (+ 425us) write_op.cc:270] Start()
0808 13:35:40.570376 (+ 77us) write_op.cc:276] Timestamp: P: 1786196105768325 usec, L: 0
0808 13:35:40.578891 (+ 8515us) write_op.cc:270] Start()
0808 13:35:40.579079 (+ 188us) write_op.cc:276] Timestamp: P: 1786196105782294 usec, L: 0
0808 13:35:40.579720 (+ 641us) write_op.cc:270] Start()
0808 13:35:40.579801 (+ 81us) write_op.cc:276] Timestamp: P: 1786196105787706 usec, L: 0
0808 13:35:40.580250 (+ 449us) write_op.cc:270] Start()
0808 13:35:40.580331 (+ 81us) write_op.cc:276] Timestamp: P: 1786196105804819 usec, L: 0
0808 13:35:40.580750 (+ 419us) write_op.cc:270] Start()
0808 13:35:40.580825 (+ 75us) write_op.cc:276] Timestamp: P: 1786196105837322 usec, L: 0
0808 13:35:40.581292 (+ 467us) write_op.cc:270] Start()
0808 13:35:40.581369 (+ 77us) write_op.cc:276] Timestamp: P: 1786196105839765 usec, L: 0
0808 13:35:40.581783 (+ 414us) write_op.cc:270] Start()
0808 13:35:40.581860 (+ 77us) write_op.cc:276] Timestamp: P: 1786196105971605 usec, L: 0
0808 13:35:40.586445 (+ 4585us) write_op.cc:270] Start()
0808 13:35:40.586576 (+ 131us) write_op.cc:276] Timestamp: P: 1786196106290488 usec, L: 0
0808 13:35:40.588082 (+ 1506us) write_op.cc:270] Start()
0808 13:35:40.588200 (+ 118us) write_op.cc:276] Timestamp: P: 1786196106361810 usec, L: 0
0808 13:35:40.589202 (+ 1002us) write_op.cc:270] Start()
0808 13:35:40.589318 (+ 116us) write_op.cc:276] Timestamp: P: 1786196106446190 usec, L: 0
0808 13:35:40.589953 (+ 635us) write_op.cc:270] Start()
0808 13:35:40.590146 (+ 193us) write_op.cc:276] Timestamp: P: 1786196106472331 usec, L: 0
0808 13:35:40.590778 (+ 632us) write_op.cc:270] Start()
0808 13:35:40.590977 (+ 199us) write_op.cc:276] Timestamp: P: 1786196106519885 usec, L: 0
0808 13:35:40.592657 (+ 1680us) write_op.cc:270] Start()
0808 13:35:40.592781 (+ 124us) write_op.cc:276] Timestamp: P: 1786196106570309 usec, L: 0
0808 13:35:40.593328 (+ 547us) write_op.cc:270] Start()
0808 13:35:40.593422 (+ 94us) write_op.cc:276] Timestamp: P: 1786196106600285 usec, L: 0
0808 13:35:40.593931 (+ 509us) write_op.cc:270] Start()
0808 13:35:40.594031 (+ 100us) write_op.cc:276] Timestamp: P: 1786196106834963 usec, L: 0
0808 13:35:40.594536 (+ 505us) write_op.cc:270] Start()
0808 13:35:40.594655 (+ 119us) write_op.cc:276] Timestamp: P: 1786196106978089 usec, L: 0
0808 13:35:40.595266 (+ 611us) write_op.cc:270] Start()
0808 13:35:40.595380 (+ 114us) write_op.cc:276] Timestamp: P: 1786196107006882 usec, L: 0
0808 13:35:40.596083 (+ 703us) write_op.cc:270] Start()
0808 13:35:40.596202 (+ 119us) write_op.cc:276] Timestamp: P: 1786196107016151 usec, L: 0
0808 13:35:40.597170 (+ 968us) write_op.cc:270] Start()
0808 13:35:40.597278 (+ 108us) write_op.cc:276] Timestamp: P: 1786196107019989 usec, L: 0
0808 13:35:40.597806 (+ 528us) write_op.cc:270] Start()
0808 13:35:40.597898 (+ 92us) write_op.cc:276] Timestamp: P: 1786196107021701 usec, L: 0
0808 13:35:40.598520 (+ 622us) write_op.cc:270] Start()
0808 13:35:40.598601 (+ 81us) write_op.cc:276] Timestamp: P: 1786196107048017 usec, L: 0
0808 13:35:40.599085 (+ 484us) write_op.cc:270] Start()
0808 13:35:40.599206 (+ 121us) write_op.cc:276] Timestamp: P: 1786196107101338 usec, L: 0
0808 13:35:40.599940 (+ 734us) write_op.cc:270] Start()
0808 13:35:40.600044 (+ 104us) write_op.cc:276] Timestamp: P: 1786196107221721 usec, L: 0
0808 13:35:40.600565 (+ 521us) write_op.cc:270] Start()
0808 13:35:40.600650 (+ 85us) write_op.cc:276] Timestamp: P: 1786196107305782 usec, L: 0
0808 13:35:40.601166 (+ 516us) write_op.cc:270] Start()
0808 13:35:40.601277 (+ 111us) write_op.cc:276] Timestamp: P: 1786196107369574 usec, L: 0
0808 13:35:40.601767 (+ 490us) write_op.cc:270] Start()
0808 13:35:40.601852 (+ 85us) write_op.cc:276] Timestamp: P: 1786196107385335 usec, L: 0
0808 13:35:40.602373 (+ 521us) write_op.cc:270] Start()
0808 13:35:40.602455 (+ 82us) write_op.cc:276] Timestamp: P: 1786196107419850 usec, L: 0
0808 13:35:40.603049 (+ 594us) write_op.cc:270] Start()
0808 13:35:40.603147 (+ 98us) write_op.cc:276] Timestamp: P: 1786196107501904 usec, L: 0
0808 13:35:40.603786 (+ 639us) write_op.cc:270] Start()
0808 13:35:40.603890 (+ 104us) write_op.cc:276] Timestamp: P: 1786196107504793 usec, L: 0
0808 13:35:40.604439 (+ 549us) write_op.cc:270] Start()
0808 13:35:40.604539 (+ 100us) write_op.cc:276] Timestamp: P: 1786196107669293 usec, L: 0
0808 13:35:40.605718 (+ 1179us) write_op.cc:270] Start()
0808 13:35:40.606567 (+ 849us) write_op.cc:276] Timestamp: P: 1786196107695652 usec, L: 0
0808 13:35:40.607159 (+ 592us) write_op.cc:270] Start()
0808 13:35:40.607260 (+ 101us) write_op.cc:276] Timestamp: P: 1786196107773815 usec, L: 0
0808 13:35:40.607773 (+ 513us) write_op.cc:270] Start()
0808 13:35:40.607890 (+ 117us) write_op.cc:276] Timestamp: P: 1786196107890767 usec, L: 0
0808 13:35:40.608508 (+ 618us) write_op.cc:270] Start()
0808 13:35:40.608695 (+ 187us) write_op.cc:276] Timestamp: P: 1786196107893421 usec, L: 0
0808 13:35:40.609331 (+ 636us) write_op.cc:270] Start()
0808 13:35:40.609426 (+ 95us) write_op.cc:276] Timestamp: P: 1786196107951600 usec, L: 0
0808 13:35:40.609934 (+ 508us) write_op.cc:270] Start()
0808 13:35:40.610020 (+ 86us) write_op.cc:276] Timestamp: P: 1786196108051648 usec, L: 0
0808 13:35:40.610502 (+ 482us) write_op.cc:270] Start()
0808 13:35:40.610577 (+ 75us) write_op.cc:276] Timestamp: P: 1786196108101146 usec, L: 0
0808 13:35:40.611075 (+ 498us) write_op.cc:270] Start()
0808 13:35:40.611151 (+ 76us) write_op.cc:276] Timestamp: P: 1786196108153043 usec, L: 0
0808 13:35:40.611630 (+ 479us) write_op.cc:270] Start()
0808 13:35:40.611720 (+ 90us) write_op.cc:276] Timestamp: P: 1786196108213599 usec, L: 0
0808 13:35:40.612158 (+ 438us) write_op.cc:270] Start()
0808 13:35:40.612236 (+ 78us) write_op.cc:276] Timestamp: P: 1786196108277639 usec, L: 0
0808 13:35:40.612737 (+ 501us) write_op.cc:270] Start()
0808 13:35:40.612812 (+ 75us) write_op.cc:276] Timestamp: P: 1786196108360478 usec, L: 0
0808 13:35:40.613268 (+ 456us) write_op.cc:270] Start()
0808 13:35:40.613339 (+ 71us) write_op.cc:276] Timestamp: P: 1786196108416022 usec, L: 0
0808 13:35:40.613791 (+ 452us) write_op.cc:270] Start()
0808 13:35:40.613865 (+ 74us) write_op.cc:276] Timestamp: P: 1786196108543598 usec, L: 0
0808 13:35:40.614484 (+ 619us) write_op.cc:270] Start()
0808 13:35:40.614554 (+ 70us) write_op.cc:276] Timestamp: P: 1786196108593300 usec, L: 0
0808 13:35:40.641430 (+ 26876us) write_op.cc:270] Start()
0808 13:35:40.641557 (+ 127us) write_op.cc:276] Timestamp: P: 1786196108633083 usec, L: 0
0808 13:35:40.653909 (+ 12352us) write_op.cc:270] Start()
0808 13:35:40.654041 (+ 132us) write_op.cc:276] Timestamp: P: 1786196108641527 usec, L: 0
0808 13:35:40.657938 (+ 3897us) write_op.cc:270] Start()
0808 13:35:40.658022 (+ 84us) write_op.cc:276] Timestamp: P: 1786196108793712 usec, L: 0
0808 13:35:40.658473 (+ 451us) write_op.cc:270] Start()
0808 13:35:40.658536 (+ 63us) write_op.cc:276] Timestamp: P: 1786196108799931 usec, L: 0
0808 13:35:40.660316 (+ 1780us) write_op.cc:270] Start()
0808 13:35:40.660390 (+ 74us) write_op.cc:276] Timestamp: P: 1786196108934073 usec, L: 0
0808 13:35:40.662970 (+ 2580us) write_op.cc:270] Start()
0808 13:35:40.663053 (+ 83us) write_op.cc:276] Timestamp: P: 1786196108997532 usec, L: 0
0808 13:35:40.663570 (+ 517us) write_op.cc:270] Start()
0808 13:35:40.663674 (+ 104us) write_op.cc:276] Timestamp: P: 1786196109012295 usec, L: 0
0808 13:35:40.664070 (+ 396us) write_op.cc:270] Start()
0808 13:35:40.664262 (+ 192us) write_op.cc:276] Timestamp: P: 1786196109042641 usec, L: 0
0808 13:35:40.664772 (+ 510us) write_op.cc:270] Start()
0808 13:35:40.664842 (+ 70us) write_op.cc:276] Timestamp: P: 1786196109094730 usec, L: 0
0808 13:35:40.671706 (+ 6864us) write_op.cc:270] Start()
0808 13:35:40.671791 (+ 85us) write_op.cc:276] Timestamp: P: 1786196109156015 usec, L: 0
0808 13:35:40.672251 (+ 460us) write_op.cc:270] Start()
0808 13:35:40.672324 (+ 73us) write_op.cc:276] Timestamp: P: 1786196109205564 usec, L: 0
0808 13:35:40.675293 (+ 2969us) write_op.cc:270] Start()
0808 13:35:40.675379 (+ 86us) write_op.cc:276] Timestamp: P: 1786196109276937 usec, L: 0
0808 13:35:40.675814 (+ 435us) write_op.cc:270] Start()
0808 13:35:40.675878 (+ 64us) write_op.cc:276] Timestamp: P: 1786196109323190 usec, L: 0
0808 13:35:40.677616 (+ 1738us) write_op.cc:270] Start()
0808 13:35:40.677681 (+ 65us) write_op.cc:276] Timestamp: P: 1786196109348057 usec, L: 0
0808 13:35:40.678073 (+ 392us) write_op.cc:270] Start()
0808 13:35:40.678136 (+ 63us) write_op.cc:276] Timestamp: P: 1786196109349974 usec, L: 0
0808 13:35:40.678525 (+ 389us) write_op.cc:270] Start()
0808 13:35:40.678590 (+ 65us) write_op.cc:276] Timestamp: P: 1786196109534161 usec, L: 0
0808 13:35:40.678978 (+ 388us) write_op.cc:270] Start()
0808 13:35:40.679048 (+ 70us) write_op.cc:276] Timestamp: P: 1786196109542817 usec, L: 0
0808 13:35:40.679441 (+ 393us) write_op.cc:270] Start()
0808 13:35:40.679505 (+ 64us) write_op.cc:276] Timestamp: P: 1786196109551805 usec, L: 0
0808 13:35:40.679900 (+ 395us) write_op.cc:270] Start()
0808 13:35:40.679985 (+ 85us) write_op.cc:276] Timestamp: P: 1786196109645087 usec, L: 0
0808 13:35:40.680471 (+ 486us) write_op.cc:270] Start()
0808 13:35:40.680536 (+ 65us) write_op.cc:276] Timestamp: P: 1786196109752376 usec, L: 0
0808 13:35:40.681005 (+ 469us) write_op.cc:270] Start()
0808 13:35:40.681081 (+ 76us) write_op.cc:276] Timestamp: P: 1786196109873462 usec, L: 0
0808 13:35:40.681566 (+ 485us) write_op.cc:270] Start()
0808 13:35:40.681643 (+ 77us) write_op.cc:276] Timestamp: P: 1786196109923639 usec, L: 0
0808 13:35:40.682133 (+ 490us) write_op.cc:270] Start()
0808 13:35:40.699957 (+ 17824us) write_op.cc:276] Timestamp: P: 1786196110011799 usec, L: 0
0808 13:35:40.700511 (+ 554us) write_op.cc:270] Start()
0808 13:35:40.700587 (+ 76us) write_op.cc:276] Timestamp: P: 1786196110115863 usec, L: 0
0808 13:35:40.700993 (+ 406us) write_op.cc:270] Start()
0808 13:35:40.701054 (+ 61us) write_op.cc:276] Timestamp: P: 1786196110179755 usec, L: 0
0808 13:35:40.701491 (+ 437us) write_op.cc:270] Start()
0808 13:35:40.701566 (+ 75us) write_op.cc:276] Timestamp: P: 1786196110206983 usec, L: 0
0808 13:35:40.702053 (+ 487us) write_op.cc:270] Start()
0808 13:35:40.702119 (+ 66us) write_op.cc:276] Timestamp: P: 1786196110309743 usec, L: 0
0808 13:35:40.702704 (+ 585us) write_op.cc:270] Start()
0808 13:35:40.702777 (+ 73us) write_op.cc:276] Timestamp: P: 1786196110343576 usec, L: 0
0808 13:35:40.717395 (+ 14618us) write_op.cc:270] Start()
0808 13:35:40.717510 (+ 115us) write_op.cc:276] Timestamp: P: 1786196110395352 usec, L: 0
0808 13:35:40.718114 (+ 604us) write_op.cc:270] Start()
0808 13:35:40.718190 (+ 76us) write_op.cc:276] Timestamp: P: 1786196110428449 usec, L: 0
0808 13:35:40.718772 (+ 582us) write_op.cc:270] Start()
0808 13:35:40.718849 (+ 77us) write_op.cc:276] Timestamp: P: 1786196110485616 usec, L: 0
0808 13:35:40.719457 (+ 608us) write_op.cc:270] Start()
0808 13:35:40.719527 (+ 70us) write_op.cc:276] Timestamp: P: 1786196110579841 usec, L: 0
0808 13:35:40.720015 (+ 488us) write_op.cc:270] Start()
0808 13:35:40.720088 (+ 73us) write_op.cc:276] Timestamp: P: 1786196110590419 usec, L: 0
0808 13:35:40.720632 (+ 544us) write_op.cc:270] Start()
0808 13:35:40.720704 (+ 72us) write_op.cc:276] Timestamp: P: 1786196110697186 usec, L: 0
0808 13:35:40.721209 (+ 505us) write_op.cc:270] Start()
0808 13:35:40.721276 (+ 67us) write_op.cc:276] Timestamp: P: 1786196110759629 usec, L: 0
0808 13:35:40.721720 (+ 444us) write_op.cc:270] Start()
0808 13:35:40.721792 (+ 72us) write_op.cc:276] Timestamp: P: 1786196110840004 usec, L: 0
0808 13:35:40.722260 (+ 468us) write_op.cc:270] Start()
0808 13:35:40.722328 (+ 68us) write_op.cc:276] Timestamp: P: 1786196110849058 usec, L: 0
0808 13:35:40.722851 (+ 523us) write_op.cc:270] Start()
0808 13:35:40.723072 (+ 221us) write_op.cc:276] Timestamp: P: 1786196110908113 usec, L: 0
0808 13:35:40.724268 (+ 1196us) write_op.cc:270] Start()
0808 13:35:40.724412 (+ 144us) write_op.cc:276] Timestamp: P: 1786196110920192 usec, L: 0
0808 13:35:40.725066 (+ 654us) write_op.cc:270] Start()
0808 13:35:40.725197 (+ 131us) write_op.cc:276] Timestamp: P: 1786196110977964 usec, L: 0
0808 13:35:40.747497 (+ 22300us) write_op.cc:270] Start()
0808 13:35:40.747595 (+ 98us) write_op.cc:276] Timestamp: P: 1786196111063085 usec, L: 0
0808 13:35:40.748151 (+ 556us) write_op.cc:270] Start()
0808 13:35:40.748247 (+ 96us) write_op.cc:276] Timestamp: P: 1786196111094659 usec, L: 0
0808 13:35:40.760078 (+ 11831us) write_op.cc:270] Start()
0808 13:35:40.760204 (+ 126us) write_op.cc:276] Timestamp: P: 1786196111122097 usec, L: 0
0808 13:35:40.760884 (+ 680us) write_op.cc:270] Start()
0808 13:35:40.760980 (+ 96us) write_op.cc:276] Timestamp: P: 1786196111128762 usec, L: 0
0808 13:35:40.764714 (+ 3734us) write_op.cc:270] Start()
0808 13:35:40.764805 (+ 91us) write_op.cc:276] Timestamp: P: 1786196111192134 usec, L: 0
0808 13:35:40.771604 (+ 6799us) write_op.cc:270] Start()
0808 13:35:40.771723 (+ 119us) write_op.cc:276] Timestamp: P: 1786196111284364 usec, L: 0
0808 13:35:40.772301 (+ 578us) write_op.cc:270] Start()
0808 13:35:40.772380 (+ 79us) write_op.cc:276] Timestamp: P: 1786196111295215 usec, L: 0
0808 13:35:40.772884 (+ 504us) write_op.cc:270] Start()
0808 13:35:40.772965 (+ 81us) write_op.cc:276] Timestamp: P: 1786196111434769 usec, L: 0
0808 13:35:40.773450 (+ 485us) write_op.cc:270] Start()
0808 13:35:40.773553 (+ 103us) write_op.cc:276] Timestamp: P: 1786196111483087 usec, L: 0
0808 13:35:40.774096 (+ 543us) write_op.cc:270] Start()
0808 13:35:40.774172 (+ 76us) write_op.cc:276] Timestamp: P: 1786196111484985 usec, L: 0
0808 13:35:40.774861 (+ 689us) write_op.cc:270] Start()
0808 13:35:40.774939 (+ 78us) write_op.cc:276] Timestamp: P: 1786196111573109 usec, L: 0
0808 13:35:40.775429 (+ 490us) write_op.cc:270] Start()
0808 13:35:40.775529 (+ 100us) write_op.cc:276] Timestamp: P: 1786196111589023 usec, L: 0
0808 13:35:40.776021 (+ 492us) write_op.cc:270] Start()
0808 13:35:40.776108 (+ 87us) write_op.cc:276] Timestamp: P: 1786196111591120 usec, L: 0
0808 13:35:40.776578 (+ 470us) write_op.cc:270] Start()
0808 13:35:40.776657 (+ 79us) write_op.cc:276] Timestamp: P: 1786196111592982 usec, L: 0
0808 13:35:40.777104 (+ 447us) write_op.cc:270] Start()
0808 13:35:40.777184 (+ 80us) write_op.cc:276] Timestamp: P: 1786196111805630 usec, L: 0
0808 13:35:40.777636 (+ 452us) write_op.cc:270] Start()
0808 13:35:40.777714 (+ 78us) write_op.cc:276] Timestamp: P: 1786196111888116 usec, L: 0
0808 13:35:40.778131 (+ 417us) write_op.cc:270] Start()
0808 13:35:40.778207 (+ 76us) write_op.cc:276] Timestamp: P: 1786196111932043 usec, L: 0
0808 13:35:40.788325 (+ 10118us) write_op.cc:270] Start()
0808 13:35:40.788417 (+ 92us) write_op.cc:276] Timestamp: P: 1786196112051466 usec, L: 0
0808 13:35:40.788879 (+ 462us) write_op.cc:270] Start()
0808 13:35:40.788949 (+ 70us) write_op.cc:276] Timestamp: P: 1786196112056231 usec, L: 0
0808 13:35:40.789480 (+ 531us) write_op.cc:270] Start()
0808 13:35:40.789633 (+ 153us) write_op.cc:276] Timestamp: P: 1786196112059084 usec, L: 0
0808 13:35:40.790120 (+ 487us) write_op.cc:270] Start()
0808 13:35:40.790201 (+ 81us) write_op.cc:276] Timestamp: P: 1786196112137883 usec, L: 0
0808 13:35:40.790836 (+ 635us) write_op.cc:270] Start()
0808 13:35:40.790937 (+ 101us) write_op.cc:276] Timestamp: P: 1786196112168336 usec, L: 0
0808 13:35:40.792607 (+ 1670us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b5400030300. stack: 00007ff0fbb8b033 00007ff0fbb8af30 00007ff0fb70c534 00000000004db2b4 00000000004db249 00007ff102ea7361 00007ff102f77a86 00007ff102f6de03 00007ff102f34f96 00007ff102f34acf 00007ff102b09040 00007ff102b12536 00007ff102b1199e 00007ff103aee2e9 00007ff1022c4684 00007ff1022c4612
0808 13:35:40.793185 (+ 578us) write_op.cc:270] Start()
0808 13:35:40.793278 (+ 93us) write_op.cc:276] Timestamp: P: 1786196112219884 usec, L: 0
0808 13:35:40.795321 (+ 2043us) write_op.cc:270] Start()
0808 13:35:40.795411 (+ 90us) write_op.cc:276] Timestamp: P: 1786196112295850 usec, L: 0
0808 13:35:40.795966 (+ 555us) write_op.cc:270] Start()
0808 13:35:40.796091 (+ 125us) write_op.cc:276] Timestamp: P: 1786196112535466 usec, L: 0
0808 13:35:40.798714 (+ 2623us) write_op.cc:270] Start()
0808 13:35:40.798816 (+ 102us) write_op.cc:276] Timestamp: P: 1786196112566448 usec, L: 0
0808 13:35:40.804055 (+ 5239us) write_op.cc:270] Start()
0808 13:35:40.804143 (+ 88us) write_op.cc:276] Timestamp: P: 1786196112594953 usec, L: 0
0808 13:35:40.804763 (+ 620us) write_op.cc:270] Start()
0808 13:35:40.804847 (+ 84us) write_op.cc:276] Timestamp: P: 1786196112638065 usec, L: 0
0808 13:35:40.805430 (+ 583us) write_op.cc:270] Start()
0808 13:35:40.805520 (+ 90us) write_op.cc:276] Timestamp: P: 1786196112640182 usec, L: 0
0808 13:35:40.819580 (+ 14060us) write_op.cc:270] Start()
0808 13:35:40.819684 (+ 104us) write_op.cc:276] Timestamp: P: 1786196112655894 usec, L: 0
0808 13:35:40.820883 (+ 1199us) write_op.cc:270] Start()
0808 13:35:40.820997 (+ 114us) write_op.cc:276] Timestamp: P: 1786196112657838 usec, L: 0
0808 13:35:40.821803 (+ 806us) write_op.cc:270] Start()
0808 13:35:40.821883 (+ 80us) write_op.cc:276] Timestamp: P: 1786196112771579 usec, L: 0
0808 13:35:40.822549 (+ 666us) write_op.cc:270] Start()
0808 13:35:40.822663 (+ 114us) write_op.cc:276] Timestamp: P: 1786196112815661 usec, L: 0
0808 13:35:40.823253 (+ 590us) write_op.cc:270] Start()
0808 13:35:40.823376 (+ 123us) write_op.cc:276] Timestamp: P: 1786196112852823 usec, L: 0
0808 13:35:40.823879 (+ 503us) write_op.cc:270] Start()
0808 13:35:40.823996 (+ 117us) write_op.cc:276] Timestamp: P: 1786196112888018 usec, L: 0
0808 13:35:40.824681 (+ 685us) write_op.cc:270] Start()
0808 13:35:40.824787 (+ 106us) write_op.cc:276] Timestamp: P: 1786196112918636 usec, L: 0
0808 13:35:40.825511 (+ 724us) write_op.cc:270] Start()
0808 13:35:40.825615 (+ 104us) write_op.cc:276] Timestamp: P: 1786196112959515 usec, L: 0
0808 13:35:40.826165 (+ 550us) write_op.cc:270] Start()
0808 13:35:40.826254 (+ 89us) write_op.cc:276] Timestamp: P: 1786196113041797 usec, L: 0
0808 13:35:40.826861 (+ 607us) write_op.cc:270] Start()
0808 13:35:40.826985 (+ 124us) write_op.cc:276] Timestamp: P: 1786196113066512 usec, L: 0
0808 13:35:40.827753 (+ 768us) write_op.cc:270] Start()
0808 13:35:40.827940 (+ 187us) write_op.cc:276] Timestamp: P: 1786196113098661 usec, L: 0
0808 13:35:40.828688 (+ 748us) write_op.cc:270] Start()
0808 13:35:40.828772 (+ 84us) write_op.cc:276] Timestamp: P: 1786196113332574 usec, L: 0
0808 13:35:40.829403 (+ 631us) write_op.cc:270] Start()
0808 13:35:40.829528 (+ 125us) write_op.cc:276] Timestamp: P: 1786196113390279 usec, L: 0
0808 13:35:40.829974 (+ 446us) write_op.cc:270] Start()
0808 13:35:40.830045 (+ 71us) write_op.cc:276] Timestamp: P: 1786196113515181 usec, L: 0
0808 13:35:40.843003 (+ 12958us) write_op.cc:270] Start()
0808 13:35:40.843141 (+ 138us) write_op.cc:276] Timestamp: P: 1786196113561683 usec, L: 0
0808 13:35:40.843838 (+ 697us) write_op.cc:270] Start()
0808 13:35:40.843998 (+ 160us) write_op.cc:276] Timestamp: P: 1786196113963001 usec, L: 0
0808 13:35:40.844555 (+ 557us) write_op.cc:270] Start()
0808 13:35:40.844650 (+ 95us) write_op.cc:276] Timestamp: P: 1786196114126465 usec, L: 0
0808 13:35:40.845072 (+ 422us) write_op.cc:270] Start()
0808 13:35:40.845144 (+ 72us) write_op.cc:276] Timestamp: P: 1786196114165002 usec, L: 0
0808 13:35:40.845543 (+ 399us) write_op.cc:270] Start()
0808 13:35:40.845611 (+ 68us) write_op.cc:276] Timestamp: P: 1786196114168254 usec, L: 0
0808 13:35:40.846003 (+ 392us) write_op.cc:270] Start()
0808 13:35:40.846074 (+ 71us) write_op.cc:276] Timestamp: P: 1786196114307321 usec, L: 0
0808 13:35:40.846480 (+ 406us) write_op.cc:270] Start()
0808 13:35:40.846549 (+ 69us) write_op.cc:276] Timestamp: P: 1786196114361499 usec, L: 0
0808 13:35:40.846952 (+ 403us) write_op.cc:270] Start()
0808 13:35:40.847021 (+ 69us) write_op.cc:276] Timestamp: P: 1786196114524448 usec, L: 0
0808 13:35:40.847420 (+ 399us) write_op.cc:270] Start()
0808 13:35:40.847488 (+ 68us) write_op.cc:276] Timestamp: P: 1786196114570933 usec, L: 0
0808 13:35:40.847885 (+ 397us) write_op.cc:270] Start()
0808 13:35:40.847955 (+ 70us) write_op.cc:276] Timestamp: P: 1786196114573177 usec, L: 0
0808 13:35:40.848345 (+ 390us) write_op.cc:270] Start()
0808 13:35:40.848408 (+ 63us) write_op.cc:276] Timestamp: P: 1786196114616870 usec, L: 0
0808 13:35:40.848801 (+ 393us) write_op.cc:270] Start()
0808 13:35:40.848860 (+ 59us) write_op.cc:276] Timestamp: P: 1786196114856379 usec, L: 0
0808 13:35:40.849208 (+ 348us) write_op.cc:270] Start()
0808 13:35:40.849278 (+ 70us) write_op.cc:276] Timestamp: P: 1786196115080222 usec, L: 0
0808 13:35:40.849599 (+ 321us) write_op.cc:270] Start()
0808 13:35:40.849657 (+ 58us) write_op.cc:276] Timestamp: P: 1786196115092456 usec, L: 0
0808 13:35:40.850001 (+ 344us) write_op.cc:270] Start()
0808 13:35:40.850062 (+ 61us) write_op.cc:276] Timestamp: P: 1786196115094369 usec, L: 0
0808 13:35:40.850403 (+ 341us) write_op.cc:270] Start()
0808 13:35:40.850470 (+ 67us) write_op.cc:276] Timestamp: P: 1786196115142286 usec, L: 0
0808 13:35:40.850827 (+ 357us) write_op.cc:270] Start()
0808 13:35:40.850891 (+ 64us) write_op.cc:276] Timestamp: P: 1786196115144672 usec, L: 0
0808 13:35:40.851271 (+ 380us) write_op.cc:270] Start()
0808 13:35:40.851361 (+ 90us) write_op.cc:276] Timestamp: P: 1786196115323233 usec, L: 0
0808 13:35:40.851790 (+ 429us) write_op.cc:270] Start()
0808 13:35:40.851862 (+ 72us) write_op.cc:276] Timestamp: P: 17
I20260808 13:35:54.517815 6033 tablet_bootstrap.cc:654] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:54.524428 5551 leader_election.cc:341] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.524863 5551 leader_election.cc:302] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:54.525523 8793 raft_consensus.cc:2726] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:54.532928 6026 tablet_bootstrap.cc:654] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:54.545145 6026 tablet_bootstrap.cc:492] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:54.545655 6026 ts_tablet_manager.cc:1397] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.080s user 0.013s sys 0.000s
I20260808 13:35:54.547861 6026 raft_consensus.cc:348] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.548657 6026 raft_consensus.cc:374] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:54.548998 6026 raft_consensus.cc:729] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:54.549764 6026 consensus_queue.cc:260] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.558050 8567 tablet_loader.cc:96] loaded metadata for tablet 007c7222c7bc48329420c740b31ddfae (table default.Table_d5bb15ded3d442699c050da99887a456 [id=67953edd9d3c47b8b489f036a64c74ae])
I20260808 13:35:54.565976 6026 ts_tablet_manager.cc:1428] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260808 13:35:54.571705 8567 tablet_loader.cc:96] loaded metadata for tablet 01982f911d394aee8f4a3cd3b7110100 (table default.Table_83babd1cd9304044b4bdb5cf0ee909a4 [id=b1e1dc25e25a4075ae747315c8130723])
I20260808 13:35:54.573699 8567 tablet_loader.cc:96] loaded metadata for tablet 023fe27fc8ce4321a43a1f6037dd9cce (table default.Table_854eb2acc74c43a688e89cd2fb1f3b96 [id=65c22acaec6e40f78aa828752330ed2a])
W20260808 13:35:54.574620 5772 leader_election.cc:341] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: de6800eceb96475284fdbffb4d23e5e3
I20260808 13:35:54.575136 5772 leader_election.cc:302] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:54.575620 8567 tablet_loader.cc:96] loaded metadata for tablet 0329c348c88a4ddda0f2b90be7a4af76 (table default.Table_a1db036771e0467fb083f890fefd54e9 [id=b449f7fe2512431790a1a39a4dde3570])
I20260808 13:35:54.576064 8868 raft_consensus.cc:2726] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:54.515578 6046 consensus_queue.cc:260] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.581394 5866 raft_consensus.cc:2396] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260808 13:35:54.583956 6046 ts_tablet_manager.cc:1428] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.153s user 0.006s sys 0.000s
I20260808 13:35:54.585731 8567 tablet_loader.cc:96] loaded metadata for tablet 0553d86a78fe4cc2896efe960ba51a27 (table default.Table_d20dd87e2f844e739e8f18ec0be12963 [id=9477c15dbac14aa3a08e0b4a82097291])
I20260808 13:35:54.587610 6046 tablet_bootstrap.cc:492] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:54.589112 8567 tablet_loader.cc:96] loaded metadata for tablet 0b96fcd7da1c4e52bcad3779a23e141c (table default.Table_432ac67683bd4fb4a2db65e5369a98c2 [id=7cf1c563f7f74ddaa65c0f335b1a3579])
I20260808 13:35:54.590183 8567 tablet_loader.cc:96] loaded metadata for tablet 0bf47d3df0ac480bb753d6dc46b4d7db (table default.Table_a446b3e7bb7944b0a5b119dd6a1b6d7c [id=4bfd4a2482364643a6d15f7a638f2eaa])
I20260808 13:35:54.591133 8567 tablet_loader.cc:96] loaded metadata for tablet 0c8fda3bd1ba4669994c78e6942185f7 (table default.Table_5d7feb173847448393c48160e861f0c1 [id=308f2ec39c62416ab0d396aad04351fd])
I20260808 13:35:54.592514 8567 tablet_loader.cc:96] loaded metadata for tablet 0cb821666b9c4a93af438ccba51c0c7c (table default.Table_7e0bc5965bd54d788347d0d41c53208f [id=296621c6dba14fbc8305eca14e4396a3])
I20260808 13:35:54.593684 8567 tablet_loader.cc:96] loaded metadata for tablet 0d0018c5333b4fde8ad4ce874660eb97 (table default.Table_db29cf6251b947269c7a48cfc05ea2e5 [id=86e7a76979764ac09c964217ebe7fae4])
I20260808 13:35:54.598450 8567 tablet_loader.cc:96] loaded metadata for tablet 0db761bf06ae47fe8ab1eea6de408e6f (table default.Table_efea7e3ddc70498f94c05ddee5ba3d76 [id=c965816b949348ba98967bdc0115eefa])
I20260808 13:35:54.599545 8567 tablet_loader.cc:96] loaded metadata for tablet 0df909324ec7453f8ecd627490802c35 (table default.Table_db9615359c8549588ff9f1d557189a0d [id=29ca8ec3c8a14e8fb3db662cbc5b6907])
I20260808 13:35:54.602072 6026 tablet_bootstrap.cc:492] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:54.603513 8567 tablet_loader.cc:96] loaded metadata for tablet 14a9373000e24e75858e6d7457587264 (table default.Table_a8fa4ae765254ab895c787d8fcc8f5ca [id=b45f6e0be7474d2aa9cb57a4fa85c212])
I20260808 13:35:54.605273 8567 tablet_loader.cc:96] loaded metadata for tablet 16a055fbb45844a89991d6972619eef4 (table default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be])
I20260808 13:35:54.607090 8567 tablet_loader.cc:96] loaded metadata for tablet 16a6552c871b4b81b6520cb084d359a2 (table default.Table_eeada3d5a8ba4dc4b96d74955c23f2bc [id=2fa9c2e807544cc8916c41200a0691cf])
I20260808 13:35:54.608137 8567 tablet_loader.cc:96] loaded metadata for tablet 1947f03e107447bb8e23fe5deaf1ea60 (table default.Table_d9c69f374e33451f843637e7cf776be0 [id=a478bcf780ae485ead557bca398e5b8a])
I20260808 13:35:54.609197 8567 tablet_loader.cc:96] loaded metadata for tablet 1950961bbaee468c8b16a764cf37f969 (table default.Table_5668bb8655e949f494b5c2e988ae33d5 [id=951c9bdf311a42a0b6fd001ef990e319])
I20260808 13:35:54.611268 8567 tablet_loader.cc:96] loaded metadata for tablet 1b9e8d48c55349058cf80bf7c2e149e7 (table default.Table_763498da5cad48e88d04c14346559749 [id=508f3453eb9f46b2ba759f89e6de84a9])
I20260808 13:35:54.614230 8567 tablet_loader.cc:96] loaded metadata for tablet 1de317c48eaa43dca038410840483f52 (table default.Table_2d152c1c176a4bc58ff1f37b2a9bfd0c [id=8f74146d8e454203a91ce42acd6a5233])
I20260808 13:35:54.616809 8567 tablet_loader.cc:96] loaded metadata for tablet 1f35509a5e584dd6a640eefa9020d8fd (table default.Table_f998dcf284c249d7a6db0e4b803a157a [id=dd464b6ca5eb49499b0921fa2b99cb29])
I20260808 13:35:54.617604 6026 tablet_bootstrap.cc:654] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:54.619433 8567 tablet_loader.cc:96] loaded metadata for tablet 205532710f0148f38d12a485bc0107ec (table default.Table_ec97d7d5adc245a6b9d4224e8ab75cd5 [id=fd9c3138231f42b791685b0df92ac968])
I20260808 13:35:54.621467 8567 tablet_loader.cc:96] loaded metadata for tablet 228ad48e495b45a29e96d427b7098497 (table default.Table_f1ad878382f04d038b6b08429d6b9db9 [id=d84314027def49d380b8c13a0431f39e])
I20260808 13:35:54.622685 8567 tablet_loader.cc:96] loaded metadata for tablet 2330be58e9d6414da65b4434c2d3b461 (table default.Table_c23b9f32680f412fbc697e136022e69c [id=7b7b344f5dda412389a0223f88351783])
I20260808 13:35:54.626235 8793 raft_consensus.cc:482] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:54.626760 8793 raft_consensus.cc:504] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.629806 8793 leader_election.cc:288] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:54.630118 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7e8060cab44121acf6dadd2c2873b3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:54.631417 5551 leader_election.cc:341] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.633224 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7e8060cab44121acf6dadd2c2873b3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:54.634858 5550 leader_election.cc:341] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.635217 5550 leader_election.cc:302] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:54.637089 8858 raft_consensus.cc:2726] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:54.643994 8654 raft_consensus.cc:482] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 946c9d8462644deba5b45fa724f1923c)
I20260808 13:35:54.644550 8654 raft_consensus.cc:504] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.646561 8654 leader_election.cc:288] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:54.653497 5772 leader_election.cc:302] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:54.660295 8654 raft_consensus.cc:2726] T 42e36d643850411498c23699835f55a6 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:54.660858 8567 tablet_loader.cc:96] loaded metadata for tablet 234591e698f8472e90e1f610da37ff3a (table default.Table_7145b6a6927345f4b8650878e403d7d3 [id=3bb853ec147643b2909c099524a4d537])
W20260808 13:35:54.661480 5771 leader_election.cc:341] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Not found: Tablet not found: daa59a18ae624bc8975bee2c460895a7
I20260808 13:35:54.663707 8567 tablet_loader.cc:96] loaded metadata for tablet 24096c0c30da4f2ea643fcdd5791a927 (table default.Table_cd53ce0f4bbd47f5a510f440e7e066c8 [id=59ed02534c50457ab73d5c2757bda959])
I20260808 13:35:54.665490 8567 tablet_loader.cc:96] loaded metadata for tablet 249dcaf769934b75945d79d827a96be6 (table default.Table_0ae56405f82f446dbd5ed65b0062fc62 [id=6a9d5e1c30ff4f5490feb1eb08805fee])
I20260808 13:35:54.666471 5772 leader_election.cc:302] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:54.674866 8567 tablet_loader.cc:96] loaded metadata for tablet 261e45848f424a349b0c5290954c5bc6 (table default.Table_da97795db35e47d8aed30d142ba853d2 [id=5d87520fa77a4254878c3f2d49f6592b])
I20260808 13:35:54.684540 8654 raft_consensus.cc:2781] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:54.684885 8567 tablet_loader.cc:96] loaded metadata for tablet 275b6aeaad044ea1a16ecd82f63dc59f (table default.Table_d2c33a8a16d34fb0a22e445a3c1be52a [id=31d10fd1087f45349072cd842a68c4a0])
I20260808 13:35:54.685220 8654 raft_consensus.cc:686] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:54.686151 8654 consensus_queue.cc:237] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.692137 8773 raft_consensus.cc:482] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:54.692649 8773 raft_consensus.cc:504] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.695204 8773 leader_election.cc:288] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:54.696204 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2be108f26bb48d49216c6b7eef6b9b8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:54.697444 5550 leader_election.cc:341] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.690407 8567 tablet_loader.cc:96] loaded metadata for tablet 2bb8c7eeec164485a4e5bb80f9d5fd72 (table default.Table_62ad61e62a5b49b480ec08bc990efa85 [id=8e1d239d7f7446fa89184bf11ddb88d9])
I20260808 13:35:54.700645 8567 tablet_loader.cc:96] loaded metadata for tablet 2d4a2d7ddb45419bbbe5df01a5e1d40f (table default.Table_fbaf65bcf1fd48249596df8d79af088b [id=b529cc209a934207ab4940d51ee4876b])
I20260808 13:35:54.702095 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2be108f26bb48d49216c6b7eef6b9b8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:54.704679 8567 tablet_loader.cc:96] loaded metadata for tablet 2d8e4af746774747a70d425068e8dd63 (table default.Table_938ca078ec5e4411b4be33056272983c [id=95f59bd3447c44c291270c448f53d78b])
W20260808 13:35:54.707861 5551 leader_election.cc:341] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:54.708256 5551 leader_election.cc:302] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:54.709429 8773 raft_consensus.cc:2726] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:54.711540 8567 tablet_loader.cc:96] loaded metadata for tablet 2e307a9ea77148c4a74385ace4bfeb73 (table default.Table_c171ae1c3b234b4e8b6f3def21f27924 [id=efc0b7748878445c8245bcd3e9657abe])
I20260808 13:35:54.712620 8567 tablet_loader.cc:96] loaded metadata for tablet 2e661b45b9b54a81aaa63b3803636566 (table default.Table_e9d344caa3c2448784d6108b083eca10 [id=6697c24737964dc09cde7d28b2e12447])
I20260808 13:35:54.719699 6026 tablet_bootstrap.cc:492] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:54.720192 6026 ts_tablet_manager.cc:1397] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.118s user 0.009s sys 0.000s
I20260808 13:35:54.740298 8567 tablet_loader.cc:96] loaded metadata for tablet 2fa86f45770d44e7b993b240607ef2fc (table default.Table_b9063cf92547431f9c63a652c79fbaae [id=53aad8e2f83c4f8d87add3777f909fa5])
I20260808 13:35:54.748976 8567 tablet_loader.cc:96] loaded metadata for tablet 3024717477814906a3d861df35e4cb6c (table default.Table_e4788552c1a8457a8d34951f6827c5e4 [id=a14f97b5af3741bebd7d88f5602675f1])
I20260808 13:35:54.750207 8567 tablet_loader.cc:96] loaded metadata for tablet 30691adffb1f40bdaeaa24f978a38bd7 (table default.Table_c2023a99a0654190adb8ada69a32f740 [id=054085276211478a8982d48b0599a684])
I20260808 13:35:54.751361 8567 tablet_loader.cc:96] loaded metadata for tablet 315c8b665cc9490da0b10623b9963154 (table default.Table_94d7e54302ae4463a43a530d34d4f264 [id=062b1b88fea4482fa36b7456f6534f4f])
I20260808 13:35:54.752766 6033 tablet_bootstrap.cc:492] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:54.753306 6033 ts_tablet_manager.cc:1397] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.414s user 0.010s sys 0.005s
I20260808 13:35:54.751174 6026 raft_consensus.cc:348] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.755977 6033 raft_consensus.cc:348] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.756692 6033 raft_consensus.cc:374] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:54.756996 6033 raft_consensus.cc:729] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
W20260808 13:35:54.759924 5283 leader_election.cc:341] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: 86c2b34c95244472beec43643e60ff5f
I20260808 13:35:54.760337 5283 leader_election.cc:302] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:54.761165 8567 tablet_loader.cc:96] loaded metadata for tablet 342027968b884f1cbc78b1371318dfc9 (table default.Table_6ea0a150e9a245babc374b7222ad1ca7 [id=c69587f42e4b437aadf52a706e37e070])
I20260808 13:35:54.761536 8744 raft_consensus.cc:2726] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:54.762315 5283 leader_election.cc:302] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:54.763288 8567 tablet_loader.cc:96] loaded metadata for tablet 344828b16cfc47d5b419f8b569055fa4 (table default.Table_9d2867cdebf84440b7aa21a71faba9c4 [id=0ddf25cf69f449cf8c7516cb84736b3d])
I20260808 13:35:54.763550 8744 raft_consensus.cc:2726] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 13:35:54.756868 6026 raft_consensus.cc:374] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:54.765671 6026 raft_consensus.cc:729] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:54.766103 8567 tablet_loader.cc:96] loaded metadata for tablet 356e15bbcd26409a9d6302599816e15f (table default.Table_322ce0f117834a81978e588b16196c2a [id=9651f57557304d8eb476787530d19f39])
I20260808 13:35:54.768981 8698 ts_tablet_manager.cc:1918] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260808 13:35:54.769471 8698 log.cc:1196] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/3a27517560d64a3b8da2cf55980e18a8
I20260808 13:35:54.769851 8837 consensus_queue.cc:1035] T 42e36d643850411498c23699835f55a6 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:54.770418 8698 ts_tablet_manager.cc:1939] T 3a27517560d64a3b8da2cf55980e18a8 P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:35:54.776695 8698 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 494) took 3215 ms. Trace:
W20260808 13:35:54.776924 7535 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 81874.2us
I20260808 13:35:54.777269 8698 rpcz_store.cc:276] 0808 13:35:51.557289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.558242 (+ 953us) service_pool.cc:227] Handling call
0808 13:35:54.381739 (+2823497us) ts_tablet_manager.cc:1145] Deleting tablet 3a27517560d64a3b8da2cf55980e18a8
0808 13:35:54.407076 (+ 25337us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:54.729375 (+322299us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:54.768717 (+ 39342us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:54.773042 (+ 4325us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44134,"tablet-delete.queue_time_us":2782291}
I20260808 13:35:54.784796 8698 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46328 (request call id 490) took 3228 ms. Trace:
I20260808 13:35:54.784822 4820 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet 3a27517560d64a3b8da2cf55980e18a8 (table default.Table_0a3697cdad3740a1baa47b3075b5e1f9 [id=19a3a9ca647c4e069a20960c7f4cc57a]) successfully deleted
I20260808 13:35:54.785320 8698 rpcz_store.cc:276] 0808 13:35:51.555991 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.607300 (+ 51309us) service_pool.cc:227] Handling call
0808 13:35:54.784384 (+3177084us) ts_tablet_manager.cc:1145] Deleting tablet 196d80d806b0432cacf2ca6aa0da8c6b
0808 13:35:54.784454 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:54.784730 (+ 276us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3105270}
W20260808 13:35:54.786762 4820 catalog_manager.cc:4808] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 196d80d806b0432cacf2ca6aa0da8c6b because the tablet was not found. No further retry: Not found: Tablet not found: 196d80d806b0432cacf2ca6aa0da8c6b
I20260808 13:35:54.795315 8567 tablet_loader.cc:96] loaded metadata for tablet 37495b29698940edb2924eb612e94bd7 (table default.Table_45fdb943ae1b499e98db51a5ee6589a2 [id=ed0521e8ecae4c609b2a7426e5a565c0])
I20260808 13:35:54.827219 6046 tablet_bootstrap.cc:654] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:54.830416 8567 tablet_loader.cc:96] loaded metadata for tablet 3999ebf74e2e4c3abae914fb3fbe3b15 (table default.Table_b8623fa3d1eb479a96e59b16c329e590 [id=c0f00fb75b204087bd476376ddfdc046])
I20260808 13:35:54.835247 8837 raft_consensus.cc:482] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:54.835934 8837 raft_consensus.cc:504] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.838380 8837 leader_election.cc:288] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:54.779714 6026 consensus_queue.cc:260] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:54.867530 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c41483119a442eb46816aa34f7f319" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:54.868155 5630 raft_consensus.cc:2454] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:54.872428 8858 raft_consensus.cc:482] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:54.874517 5285 leader_election.cc:302] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:54.877454 8837 raft_consensus.cc:2781] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:54.877806 8837 raft_consensus.cc:482] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:54.878103 8837 raft_consensus.cc:3037] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:54.898990 6033 consensus_queue.cc:260] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.901309 6033 ts_tablet_manager.cc:1428] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.148s user 0.005s sys 0.001s
I20260808 13:35:54.922665 8567 tablet_loader.cc:96] loaded metadata for tablet 3ab078df59e34e1296526175d74e38f7 (table default.Table_d39fd423849c45c69b7338e8c6a87c30 [id=f5bbd5d961794ddcab1f07f3931ab031])
I20260808 13:35:54.923969 8567 tablet_loader.cc:96] loaded metadata for tablet 3ab26fa1872e4acf874f6577bded5e43 (table default.Table_98d1673dbc6c43a78c1353fb1b151feb [id=cabb2a6901e249b39ba549cdb2be0f88])
I20260808 13:35:54.925128 8567 tablet_loader.cc:96] loaded metadata for tablet 3b1a27f687b54bc592ef4efb72b7247f (table default.Table_661e11184b9346b1bbae005034dddd62 [id=0ded1d2205b248c8acf4ba8baac43096])
I20260808 13:35:54.926314 8567 tablet_loader.cc:96] loaded metadata for tablet 3d07c40bd01845ef871a9e87ef29e9b7 (table default.Table_e4f01562a52f4079b8b916b057cf8014 [id=76320b3dd2994b4587dc8a6b3a71825d])
I20260808 13:35:54.927429 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c41483119a442eb46816aa34f7f319" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:54.928025 5878 raft_consensus.cc:2454] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:54.886405 8837 raft_consensus.cc:504] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.872905 8858 raft_consensus.cc:504] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:54.944464 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c41483119a442eb46816aa34f7f319" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:54.945148 5635 raft_consensus.cc:3037] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:54.973973 8858 leader_election.cc:288] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
W20260808 13:35:54.967793 5772 leader_election.cc:341] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: daa59a18ae624bc8975bee2c460895a7
W20260808 13:35:54.983007 4820 leader_election.cc:334] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555): Timed out: RequestConsensusVote RPC to 127.4.170.61:46555 timed out after 0.742s (SENT)
I20260808 13:35:54.983480 4820 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:35:54.977882 5772 leader_election.cc:302] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:54.984531 8815 raft_consensus.cc:2726] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:35:54.946391 8837 leader_election.cc:288] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:54.985831 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f778e7fa51414cb209fbe1aea92739" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:54.939891 8684 ts_tablet_manager.cc:1918] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260808 13:35:54.999611 8684 log.cc:1196] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/7e8cf06462484c99a860762a82f7ad64
I20260808 13:35:54.977487 6046 tablet_bootstrap.cc:492] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:55.000492 8684 ts_tablet_manager.cc:1939] T 7e8cf06462484c99a860762a82f7ad64 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:55.000797 6046 ts_tablet_manager.cc:1397] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.413s user 0.014s sys 0.008s
I20260808 13:35:55.003322 6046 raft_consensus.cc:348] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.003988 6046 raft_consensus.cc:374] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.004243 6046 raft_consensus.cc:729] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:35:55.005108 5551 leader_election.cc:341] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.006229 5635 raft_consensus.cc:2454] T b1c41483119a442eb46816aa34f7f319 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:55.007349 5285 leader_election.cc:302] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:55.009260 8744 raft_consensus.cc:2781] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:55.009934 8744 raft_consensus.cc:686] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:55.002380 6033 tablet_bootstrap.cc:492] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:55.010749 8744 consensus_queue.cc:237] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.015868 6033 tablet_bootstrap.cc:654] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.016444 6026 ts_tablet_manager.cc:1428] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.296s user 0.010s sys 0.007s
I20260808 13:35:55.017551 6026 tablet_bootstrap.cc:492] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:55.018678 8793 raft_consensus.cc:482] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.019150 8793 raft_consensus.cc:504] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.021088 8793 leader_election.cc:288] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:55.024533 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea8f5322a404837b91e1d96749926b7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:55.027765 5551 leader_election.cc:341] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.031831 8793 raft_consensus.cc:482] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.032354 8793 raft_consensus.cc:504] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.034370 8793 leader_election.cc:288] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:55.036011 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d2aae6405349fc887ec8527464e46a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:55.039637 5551 leader_election.cc:341] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.046044 6046 consensus_queue.cc:260] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.051223 6046 ts_tablet_manager.cc:1428] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.050s user 0.006s sys 0.000s
I20260808 13:35:55.020977 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f778e7fa51414cb209fbe1aea92739" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:55.028990 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 61230.2us
I20260808 13:35:55.037964 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea8f5322a404837b91e1d96749926b7" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:55.038817 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d2aae6405349fc887ec8527464e46a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:55.060086 5550 leader_election.cc:341] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.060513 5550 leader_election.cc:302] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:55.061440 8858 raft_consensus.cc:2726] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:55.062698 5550 leader_election.cc:341] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.063103 5550 leader_election.cc:302] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:55.022233 8903 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:35:55.065759 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c41483119a442eb46816aa34f7f319" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:55.066315 5877 raft_consensus.cc:3037] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:55.066854 6046 tablet_bootstrap.cc:492] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:55.067075 6026 tablet_bootstrap.cc:654] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.074493 6046 tablet_bootstrap.cc:654] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.080369 8858 raft_consensus.cc:2726] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:55.080838 5550 leader_election.cc:341] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.081259 5550 leader_election.cc:302] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:55.081378 8858 raft_consensus.cc:482] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.081808 8858 raft_consensus.cc:504] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.081030 6046 tablet_bootstrap.cc:492] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:55.083007 6046 ts_tablet_manager.cc:1397] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.000s
I20260808 13:35:55.083900 8567 tablet_loader.cc:96] loaded metadata for tablet 41347e0da1654dde8b48c896db6a0915 (table default.Table_280edf174ae84743aac082cfd2788315 [id=ae895d0078c748d78940cf2152e0e635])
I20260808 13:35:55.084144 8793 raft_consensus.cc:2726] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.085179 8567 tablet_loader.cc:96] loaded metadata for tablet 42218e4330d147f389717108bfe0acea (table default.Table_660efdc735cf4c31bf966be22b38dc28 [id=f603e442844e4a09bfecb79b93014265])
I20260808 13:35:55.085259 6046 raft_consensus.cc:348] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.085901 6046 raft_consensus.cc:374] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.086177 6046 raft_consensus.cc:729] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:55.086309 8567 tablet_loader.cc:96] loaded metadata for tablet 42e36d643850411498c23699835f55a6 (table default.Table_5128302bae8343618fb6509466b51c71 [id=5c3825e556e04b17b0437380aa662748])
I20260808 13:35:55.091639 8684 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 477) took 3857 ms. Trace:
I20260808 13:35:55.092072 8684 rpcz_store.cc:276] 0808 13:35:51.234696 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.240530 (+ 5834us) service_pool.cc:227] Handling call
0808 13:35:54.320102 (+3079572us) ts_tablet_manager.cc:1145] Deleting tablet 7e8cf06462484c99a860762a82f7ad64
0808 13:35:54.320230 (+ 128us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:54.725157 (+404927us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:54.939598 (+214441us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:55.091565 (+151967us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":80468,"tablet-delete.queue_time_us":3078935}
I20260808 13:35:55.093372 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:55.093568 8858 leader_election.cc:288] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:55.093706 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 7e8cf06462484c99a860762a82f7ad64 (table default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]) successfully deleted
I20260808 13:35:55.094758 8684 tablet_replica.cc:330] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:55.094653 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:55.094825 5551 leader_election.cc:341] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.095709 8684 raft_consensus.cc:2229] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:55.096243 8684 raft_consensus.cc:2258] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:55.095781 5877 raft_consensus.cc:2454] T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
W20260808 13:35:55.097777 5550 leader_election.cc:341] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.098176 5550 leader_election.cc:302] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:55.098929 8858 raft_consensus.cc:2726] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.128435 6026 tablet_bootstrap.cc:492] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:55.128943 6026 ts_tablet_manager.cc:1397] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.112s user 0.013s sys 0.008s
I20260808 13:35:55.087568 6046 consensus_queue.cc:260] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.133687 6026 raft_consensus.cc:348] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.135988 8793 raft_consensus.cc:482] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.141896 6026 raft_consensus.cc:374] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.142168 6026 raft_consensus.cc:729] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:55.142036 8793 raft_consensus.cc:504] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.144299 8793 leader_election.cc:288] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:55.145325 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d68ab73bae4dd9af46e907b1fd198c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:55.146724 5550 leader_election.cc:341] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.147522 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d68ab73bae4dd9af46e907b1fd198c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:55.139106 6046 ts_tablet_manager.cc:1428] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.056s user 0.005s sys 0.001s
W20260808 13:35:55.149077 5551 leader_election.cc:341] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.149505 5551 leader_election.cc:302] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:55.150677 8793 raft_consensus.cc:2726] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.154807 8868 raft_consensus.cc:482] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.155370 8868 raft_consensus.cc:504] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.157394 8868 leader_election.cc:288] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:55.159482 6033 tablet_bootstrap.cc:492] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:55.159920 6033 ts_tablet_manager.cc:1397] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.158s user 0.010s sys 0.000s
I20260808 13:35:55.160256 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba5fa465ec04bbb801a821a71a8f87d" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:55.163039 8684 ts_tablet_manager.cc:1905] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:55.162444 6033 raft_consensus.cc:348] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.174217 6033 raft_consensus.cc:374] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.174528 6033 raft_consensus.cc:729] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:55.175256 6033 consensus_queue.cc:260] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.210191 8793 raft_consensus.cc:482] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.210759 8793 raft_consensus.cc:504] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.212996 8793 leader_election.cc:288] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:55.214145 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3e9c6770fd402290b45eed85de6f6f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:55.216284 5550 leader_election.cc:341] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.220547 6033 ts_tablet_manager.cc:1428] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.060s user 0.007s sys 0.000s
I20260808 13:35:55.240511 6033 tablet_bootstrap.cc:492] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:55.242437 8793 raft_consensus.cc:482] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.242977 8793 raft_consensus.cc:504] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.245092 8793 leader_election.cc:288] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:55.247184 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3424ca753bcc4f448d314c56aba3323e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:55.255591 5550 leader_election.cc:341] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.262565 6033 tablet_bootstrap.cc:654] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.296990 8793 raft_consensus.cc:482] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.297487 8793 raft_consensus.cc:504] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.299463 8793 leader_election.cc:288] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:55.301047 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f53e5b5c4bd4a9385ad198b283c6e79" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:55.302403 5550 leader_election.cc:341] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.331355 8684 ts_tablet_manager.cc:1918] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260808 13:35:55.352253 8793 raft_consensus.cc:482] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.352754 8793 raft_consensus.cc:504] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.354904 8793 leader_election.cc:288] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:55.357425 8684 log.cc:1196] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/3a27517560d64a3b8da2cf55980e18a8
I20260808 13:35:55.358274 8684 ts_tablet_manager.cc:1939] T 3a27517560d64a3b8da2cf55980e18a8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:55.356719 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20113491b108476b95b8f32b69176d0e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:55.359669 5550 leader_election.cc:341] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.361640 6033 tablet_bootstrap.cc:492] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:55.362059 6033 ts_tablet_manager.cc:1397] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.122s user 0.004s sys 0.011s
I20260808 13:35:55.364393 6033 raft_consensus.cc:348] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.365001 6033 raft_consensus.cc:374] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.365240 6033 raft_consensus.cc:729] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:55.162132 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba5fa465ec04bbb801a821a71a8f87d" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:55.214712 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3e9c6770fd402290b45eed85de6f6f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:55.365808 6033 consensus_queue.cc:260] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.249190 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3424ca753bcc4f448d314c56aba3323e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:55.366716 5771 leader_election.cc:302] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:55.367115 5551 leader_election.cc:341] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.367519 5551 leader_election.cc:302] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:55.300633 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f53e5b5c4bd4a9385ad198b283c6e79" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:55.368561 5551 leader_election.cc:341] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.349764 8833 raft_consensus.cc:482] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.368938 5551 leader_election.cc:302] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:55.369088 8833 raft_consensus.cc:504] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:55.369915 5551 leader_election.cc:341] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.370225 5551 leader_election.cc:302] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:55.358003 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20113491b108476b95b8f32b69176d0e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:55.372224 8858 raft_consensus.cc:2726] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.372814 8858 raft_consensus.cc:2726] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.372819 8833 leader_election.cc:288] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:55.375115 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6800eceb96475284fdbffb4d23e5e3" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:55.375444 8654 raft_consensus.cc:2726] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.375689 5877 raft_consensus.cc:2454] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:55.381930 8567 tablet_loader.cc:96] loaded metadata for tablet 433fcbc6cdca458b8404ae5862f280a3 (table default.Table_e4b2b77a6c9343b39de382dcd433c9e7 [id=6fdd797fbbf94dafa4f0048d8f4fb099])
I20260808 13:35:55.382694 8793 raft_consensus.cc:2726] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.383852 8567 tablet_loader.cc:96] loaded metadata for tablet 4561c9af06e14ed8a2ec4c7285e2ad99 (table default.Table_0a849c19427549aea0eabc1e0324725f [id=f58086e90fc34ca3a5d31417a82dad6c])
W20260808 13:35:55.374709 5551 leader_election.cc:341] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.385157 5551 leader_election.cc:302] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:35:55.386442 5285 leader_election.cc:341] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: de6800eceb96475284fdbffb4d23e5e3
W20260808 13:35:55.387270 7535 leader_election.cc:334] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555): Timed out: RequestConsensusVote RPC to 127.4.170.61:46555 timed out after 0.602s (SENT)
I20260808 13:35:55.387606 8793 raft_consensus.cc:2726] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.387790 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:55.387848 8567 tablet_loader.cc:96] loaded metadata for tablet 4747a38c520d40848f4d3c261499aa88 (table default.Table_e024f29e457447139994e4d9c43a41ce [id=334643ef0e864267b0f17674d4f1c06c])
I20260808 13:35:55.389691 8567 tablet_loader.cc:96] loaded metadata for tablet 48e8293aa32144bcb89047096c88ffec (table default.Table_9b62f270f0aa43bf92edae3c1e5b0ffd [id=99cf3d4080734790a0f96ac8b1239008])
I20260808 13:35:55.391002 8567 tablet_loader.cc:96] loaded metadata for tablet 4b7284afe0294644a368cf51451c7ab6 (table default.Table_07a60e9e6b984fbd97ce35f3f18b895a [id=26fb789fff434bb0825daf68d5378013])
I20260808 13:35:55.392217 5283 leader_election.cc:302] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:55.393184 8841 raft_consensus.cc:2781] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:55.393549 8841 raft_consensus.cc:482] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:55.393870 8841 raft_consensus.cc:3037] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:55.394030 6033 ts_tablet_manager.cc:1428] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.032s user 0.006s sys 0.001s
I20260808 13:35:55.394747 8684 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 481) took 4141 ms. Trace:
I20260808 13:35:55.395102 8684 rpcz_store.cc:276] 0808 13:35:51.252878 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.253012 (+ 134us) service_pool.cc:227] Handling call
0808 13:35:55.094532 (+3841520us) ts_tablet_manager.cc:1145] Deleting tablet 3a27517560d64a3b8da2cf55980e18a8
0808 13:35:55.094592 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:55.228889 (+134297us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:55.331099 (+102210us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:55.394677 (+ 63578us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":517,"tablet-delete.queue_time_us":3840906}
I20260808 13:35:55.397091 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 3a27517560d64a3b8da2cf55980e18a8 (table default.Table_0a3697cdad3740a1baa47b3075b5e1f9 [id=19a3a9ca647c4e069a20960c7f4cc57a]) successfully deleted
I20260808 13:35:55.405320 5877 raft_consensus.cc:3037] T 1ba5fa465ec04bbb801a821a71a8f87d P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:55.409394 8684 tablet_replica.cc:330] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:55.409818 6033 tablet_bootstrap.cc:492] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:55.410205 6046 tablet_bootstrap.cc:492] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:55.411511 8567 tablet_loader.cc:96] loaded metadata for tablet 519b2815fd4e4a3691037670c9f61418 (table default.Table_e70822ac71fa4d0e9205367df025ad3a [id=b809e4f7fa074a608bceba5f738df6a6])
I20260808 13:35:55.413920 8567 tablet_loader.cc:96] loaded metadata for tablet 52deb2e968c04b99a436e04ae78d9969 (table default.Table_9f6ee103b59c4a17b37f857fbb83c22c [id=2256103e25f04641bd14f96720840445])
I20260808 13:35:55.415118 6026 consensus_queue.cc:260] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.428767 6033 tablet_bootstrap.cc:654] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.436094 8567 tablet_loader.cc:96] loaded metadata for tablet 52e8243692d847039aafda3e0d0095fb (table default.Table_01763d5cef124bad885eaaba0686787b [id=8269e8c2637f41859e8ead761bc2d7bc])
I20260808 13:35:55.437425 8567 tablet_loader.cc:96] loaded metadata for tablet 538d9a883e9d46668fde30876352d20e (table default.Table_41f3478fe8c94fbfaf7fcbdee22ce987 [id=e3e121c36ff44ea0a3a98185c8cce381])
I20260808 13:35:55.457026 8567 tablet_loader.cc:96] loaded metadata for tablet 54cd7746bb6a4252b74517999782bbab (table default.Table_0f52af74f5c34a0aa2f84a521c188673 [id=2378817b60944a789ff39c84ebdc8991])
I20260808 13:35:55.460919 8858 raft_consensus.cc:482] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.461385 8858 raft_consensus.cc:504] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.463321 8858 leader_election.cc:288] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:55.465407 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3272708a835c432c878cc330e44490c5" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:55.466892 5551 leader_election.cc:341] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.467703 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3272708a835c432c878cc330e44490c5" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:55.468796 5550 leader_election.cc:341] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.469154 5550 leader_election.cc:302] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:55.470244 8858 raft_consensus.cc:2726] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.478448 8567 tablet_loader.cc:96] loaded metadata for tablet 55ca543166ab479590ca4e38289024cd (table default.Table_6c7876f6c0ea460d8fa5cd00a7baee97 [id=2d102da12b5240a59c2ab0bcacb2c405])
I20260808 13:35:55.479669 8567 tablet_loader.cc:96] loaded metadata for tablet 5663fa6a1bb148c3afc9340477b05a4b (table default.Table_b8b2109694134a6bbc7c2bc43b158e1c [id=9720988d247f4a2fbd324a101ed21590])
I20260808 13:35:55.489506 6033 tablet_bootstrap.cc:492] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:55.489966 6033 ts_tablet_manager.cc:1397] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.080s user 0.009s sys 0.000s
I20260808 13:35:55.492276 6033 raft_consensus.cc:348] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.492959 6033 raft_consensus.cc:374] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.493197 6033 raft_consensus.cc:729] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:55.502844 6033 consensus_queue.cc:260] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.504580 6033 ts_tablet_manager.cc:1428] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.014s user 0.003s sys 0.002s
I20260808 13:35:55.508827 6026 ts_tablet_manager.cc:1428] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.380s user 0.005s sys 0.000s
I20260808 13:35:55.509775 6026 tablet_bootstrap.cc:492] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:55.516920 8567 tablet_loader.cc:96] loaded metadata for tablet 58b61e17eada43b5a6ae8a664483588d (table default.Table_1955ad58908247a3b3002dc6a8ff2336 [id=85520e0aaa3942268225520f28506cd8])
I20260808 13:35:55.428510 8841 raft_consensus.cc:504] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.522060 8567 tablet_loader.cc:96] loaded metadata for tablet 594afd4174644c218951b9df82360bfe (table default.Table_48ee76308d8846b49515e00b6bf24244 [id=11a917856b1045d1891af36f7a71e26c])
W20260808 13:35:55.528451 7535 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 141189us
I20260808 13:35:55.531373 6033 tablet_bootstrap.cc:492] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:55.536260 8841 leader_election.cc:288] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:55.537325 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6800eceb96475284fdbffb4d23e5e3" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:55.537850 5877 raft_consensus.cc:3037] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:55.542914 5285 leader_election.cc:341] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: de6800eceb96475284fdbffb4d23e5e3
I20260808 13:35:55.550124 5877 raft_consensus.cc:2454] T de6800eceb96475284fdbffb4d23e5e3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:55.551451 5283 leader_election.cc:302] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:55.552260 8833 raft_consensus.cc:2781] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:55.552729 8833 raft_consensus.cc:686] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:55.553503 8833 consensus_queue.cc:237] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.524314 8567 tablet_loader.cc:96] loaded metadata for tablet 5ae01fdfbc564fb8abf9b6bf9a308837 (table default.Table_025eb6ef4e024bec917c3272ba1b8c35 [id=7dda6de1004344a987b5ede426b5a758])
I20260808 13:35:55.558023 8567 tablet_loader.cc:96] loaded metadata for tablet 5c3535f396f74749a6a28907c6d93abd (table default.Table_953730643273433c8353604101a4d237 [id=d9baa62ea5c249a898586a45f8b4f372])
I20260808 13:35:55.558377 8912 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:35:55.554538 6033 tablet_bootstrap.cc:654] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.579490 8567 tablet_loader.cc:96] loaded metadata for tablet 603ab05a512d4ed1bcac2a5b86d6ee7d (table default.Table_0e051b613c6e452f89c46cb5aff7f7dd [id=5b28fc73db9a4600bfa8c43e9929b4cb])
I20260808 13:35:55.589514 8567 tablet_loader.cc:96] loaded metadata for tablet 604d2113451642329b877f8beff71618 (table default.Table_6c19755f3f9149e7bdac49809abbe2a3 [id=bd6ed2fb84f44308a249c7b2cfa8d6a3])
I20260808 13:35:55.592258 8567 tablet_loader.cc:96] loaded metadata for tablet 6278446e587b42168d388acdf7ec284e (table default.Table_d99de9b6e1e64b48873233ac5849ea09 [id=88fdd9e2283a4018ad2e1d02dc9cd4ea])
I20260808 13:35:55.594234 8567 tablet_loader.cc:96] loaded metadata for tablet 64888f0d27c94ee7b25c0dda57e766f6 (table default.Table_93429697aa7c4405b439864c62c45c29 [id=feca1558e3e74fb2a9de982f5a2a5302])
I20260808 13:35:55.613478 8821 consensus_queue.cc:1035] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:55.616698 6033 tablet_bootstrap.cc:492] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:55.617130 6033 ts_tablet_manager.cc:1397] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.086s user 0.009s sys 0.000s
I20260808 13:35:55.619374 6033 raft_consensus.cc:348] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.620076 6033 raft_consensus.cc:374] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.630954 6033 raft_consensus.cc:729] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:55.641772 8684 raft_consensus.cc:2229] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:55.642649 8684 raft_consensus.cc:2258] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:55.631625 6033 consensus_queue.cc:260] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.645453 8684 ts_tablet_manager.cc:1905] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:35:55.648123 6033 ts_tablet_manager.cc:1428] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.031s user 0.008s sys 0.000s
I20260808 13:35:55.651705 8567 tablet_loader.cc:96] loaded metadata for tablet 64be4bede9d04d2db0940a75e0e9b2fb (table default.Table_6b9bc590a69746909348fbcbb7856cdf [id=03f2c8cadea14860aee2cd51574379b6])
I20260808 13:35:55.653088 8567 tablet_loader.cc:96] loaded metadata for tablet 658368f1cd8d470c83c613f9697204ee (table default.Table_5f0b33d3ad8942c0adb8ded9e15220c3 [id=f432910006cd459b9c1c110e9a4a3bda])
I20260808 13:35:55.659411 6033 tablet_bootstrap.cc:492] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:55.659760 5379 raft_consensus.cc:3037] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:55.654422 8567 tablet_loader.cc:96] loaded metadata for tablet 66d985de8de040829569d1f5783c6fef (table default.Table_7a1963a007954f8ca1d330f9d1936dd3 [id=d5b4528cac46486e857db6176b4b3a2b])
I20260808 13:35:55.683408 8654 consensus_queue.cc:1035] T 7454bf27bc1a45d2af0d8f897b2f9f97 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:55.685269 8567 tablet_loader.cc:96] loaded metadata for tablet 677154c6f4584ee0bb103ab6f27fa9ea (table default.Table_465ac7f2a60c415a8809fdfccb4b5e27 [id=f5be929e60354ded904b228314f820d9])
I20260808 13:35:55.709684 6033 tablet_bootstrap.cc:654] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.719224 6033 tablet_bootstrap.cc:492] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:55.719647 6033 ts_tablet_manager.cc:1397] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.060s user 0.012s sys 0.002s
I20260808 13:35:55.721765 6033 raft_consensus.cc:348] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.722478 6033 raft_consensus.cc:374] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.722741 6033 raft_consensus.cc:729] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:55.723320 6033 consensus_queue.cc:260] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.725335 6033 ts_tablet_manager.cc:1428] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 13:35:55.741206 8567 tablet_loader.cc:96] loaded metadata for tablet 70ec24d1bd3a42b0af377a6461f9264d (table default.Table_7e9aac444b5a43f29dc7608e6ef379f3 [id=6a2ea5033865443d85ac6e6429054471])
I20260808 13:35:55.589632 6046 tablet_bootstrap.cc:654] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.754333 8654 raft_consensus.cc:482] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.754900 8654 raft_consensus.cc:504] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.768036 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09eb729339e47eb8db88eb6509daab4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:55.768738 5630 raft_consensus.cc:2454] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:55.770656 5772 leader_election.cc:302] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:55.772485 8868 raft_consensus.cc:2781] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:55.772671 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09eb729339e47eb8db88eb6509daab4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:55.772905 8868 raft_consensus.cc:482] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:55.773253 8868 raft_consensus.cc:3037] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:55.780026 8654 leader_election.cc:288] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:55.795462 6026 tablet_bootstrap.cc:654] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.801759 6033 tablet_bootstrap.cc:492] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:35:55.810258 5771 leader_election.cc:341] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.835248 8684 ts_tablet_manager.cc:1918] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:55.842224 8684 log.cc:1196] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/daa59a18ae624bc8975bee2c460895a7
I20260808 13:35:55.842854 8567 tablet_loader.cc:96] loaded metadata for tablet 72c4ddec62c242c88b3a759d8361cd30 (table default.Table_920e3ee624f44ee4b19c15e0d7fb4358 [id=844dc96c93564895911d827c23c02689])
I20260808 13:35:55.843516 8684 ts_tablet_manager.cc:1939] T daa59a18ae624bc8975bee2c460895a7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:55.844570 8567 tablet_loader.cc:96] loaded metadata for tablet 73e2c53a99ac4ce0a0e0c0c4d5ef25b3 (table default.Table_51d39ab358f3470084d7cd6e270eee0c [id=bbe62ee3dd5d494d935378ada5650363])
I20260808 13:35:55.845868 8567 tablet_loader.cc:96] loaded metadata for tablet 7454bf27bc1a45d2af0d8f897b2f9f97 (table default.Table_0616044c71e04d1a83b3ffd25b84f54e [id=71c485130d814412bd042eb5012ba511])
I20260808 13:35:55.855644 6046 tablet_bootstrap.cc:492] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:55.856089 6046 ts_tablet_manager.cc:1397] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.446s user 0.017s sys 0.008s
I20260808 13:35:55.858593 6046 raft_consensus.cc:348] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.859473 6046 raft_consensus.cc:374] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.859814 6046 raft_consensus.cc:729] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:55.860584 6046 consensus_queue.cc:260] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.863160 6046 ts_tablet_manager.cc:1428] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260808 13:35:55.864157 6046 tablet_bootstrap.cc:492] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:55.864859 8684 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 483) took 4595 ms. Trace:
I20260808 13:35:55.876960 8684 rpcz_store.cc:276] 0808 13:35:51.269681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.269814 (+ 133us) service_pool.cc:227] Handling call
0808 13:35:55.397884 (+4128070us) ts_tablet_manager.cc:1145] Deleting tablet daa59a18ae624bc8975bee2c460895a7
0808 13:35:55.409203 (+ 11319us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:55.763960 (+354757us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:55.834958 (+ 70998us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:55.864780 (+ 29822us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":231172,"tablet-delete.queue_time_us":4127430}
I20260808 13:35:55.883003 8929 raft_consensus.cc:482] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.884076 8929 raft_consensus.cc:504] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.886735 8929 leader_election.cc:288] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:55.887460 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:55.888168 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:55.888827 5630 raft_consensus.cc:2454] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:55.889040 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet daa59a18ae624bc8975bee2c460895a7 (table default.Table_1eb4e0870a824768a3ecace6bf383c25 [id=039cce4a82b84cec947b7920d21d39cc]) successfully deleted
I20260808 13:35:55.865170 6033 tablet_bootstrap.cc:654] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.890632 8684 tablet_replica.cc:330] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:35:55.890710 5772 leader_election.cc:302] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:35:55.896770 5771 leader_election.cc:341] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.896906 6033 tablet_bootstrap.cc:492] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:55.897786 8929 raft_consensus.cc:2781] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:55.898118 8929 raft_consensus.cc:482] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:55.897428 6033 ts_tablet_manager.cc:1397] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.109s user 0.008s sys 0.006s
I20260808 13:35:55.898396 8929 raft_consensus.cc:3037] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:55.900838 6033 raft_consensus.cc:348] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.901613 6033 raft_consensus.cc:374] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.901851 6033 raft_consensus.cc:729] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:55.899554 8858 raft_consensus.cc:482] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.902412 6033 consensus_queue.cc:260] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.902377 8858 raft_consensus.cc:504] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.904479 6033 ts_tablet_manager.cc:1428] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260808 13:35:55.905505 8684 raft_consensus.cc:2229] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:35:55.906046 8684 raft_consensus.cc:2258] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:35:55.906646 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61dfd47db0a74ed8b7228b559dd68459" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:55.908242 8684 ts_tablet_manager.cc:1905] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:35:55.909077 5551 leader_election.cc:341] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.911286 6033 tablet_bootstrap.cc:492] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:55.911974 8858 leader_election.cc:288] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:55.913642 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61dfd47db0a74ed8b7228b559dd68459" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:55.915369 5550 leader_election.cc:341] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:55.915751 5550 leader_election.cc:302] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:55.916365 6033 tablet_bootstrap.cc:654] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.917622 8793 raft_consensus.cc:2726] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.926949 8821 raft_consensus.cc:482] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.927508 8821 raft_consensus.cc:504] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.929623 8821 leader_election.cc:288] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:55.931528 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e3245c8ba54f69bc329a5b75115127" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:55.934141 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e3245c8ba54f69bc329a5b75115127" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:55.936298 5771 leader_election.cc:302] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:55.937722 8821 raft_consensus.cc:2726] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:55.939577 6046 tablet_bootstrap.cc:654] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:55.944137 8868 raft_consensus.cc:504] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:55.948001 6026 tablet_bootstrap.cc:492] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:55.948482 6026 ts_tablet_manager.cc:1397] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.439s user 0.009s sys 0.003s
I20260808 13:35:55.951223 6026 raft_consensus.cc:348] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.952069 6026 raft_consensus.cc:374] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:55.952382 6026 raft_consensus.cc:729] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:55.927280 8567 tablet_loader.cc:96] loaded metadata for tablet 76f6574b348a44e193245f89b67ae061 (table default.Table_4579204171cf47c6b668cedc2efaa649 [id=71918a8fb02640a48c7838e607490743])
I20260808 13:35:55.959048 6026 consensus_queue.cc:260] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.953600 8929 raft_consensus.cc:504] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.971302 8821 raft_consensus.cc:482] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:55.971855 8821 raft_consensus.cc:504] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:55.973691 8567 tablet_loader.cc:96] loaded metadata for tablet 7b128feff212456faa1769efd96f6bd8 (table default.Table_465b28acac534aa0960266bb6c7efd27 [id=087eec2212324876bebfc252a552d074])
I20260808 13:35:55.974069 8821 leader_election.cc:288] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:55.975263 8567 tablet_loader.cc:96] loaded metadata for tablet 7b2335c2182d4c2abaa254f4554fd473 (table default.Table_e916116a3ec24469b4670c87eeaac98e [id=f348502b786943fdb6bcb104e73355f4])
I20260808 13:35:55.976697 8567 tablet_loader.cc:96] loaded metadata for tablet 7c7e8060cab44121acf6dadd2c2873b3 (table default.Table_de3d9a6e588d44f698c656a21558e2ae [id=f92accd458dd439abf6c75dd1cd79df5])
I20260808 13:35:55.977361 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9e8d48c55349058cf80bf7c2e149e7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:55.978006 8567 tablet_loader.cc:96] loaded metadata for tablet 7d31516006bc4a65b496dcf363db464a (table default.Table_1c113a710bb14973be6ef54f41e550f8 [id=7929b3dde23940a8a96690fff0077013])
I20260808 13:35:55.978031 5630 raft_consensus.cc:2454] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:55.980211 5772 leader_election.cc:302] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:55.982832 8567 tablet_loader.cc:96] loaded metadata for tablet 8514ca242bb2418d8ddbb88548b21832 (table default.Table_280b5d6fc90b4748b675e41e2e7071f5 [id=a2a1f0d168074cabbd4a680e62535962])
I20260808 13:35:55.983790 8821 raft_consensus.cc:2781] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:55.987357 8821 raft_consensus.cc:482] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:55.988149 8821 raft_consensus.cc:3037] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:55.988703 8567 tablet_loader.cc:96] loaded metadata for tablet 8675dbcac3dc4392be833df89eb3b42c (table default.Table_834da6ca85d64417964c7976b57c1329 [id=ad4e3ed6750e4b7b8262ec862dd1c699])
I20260808 13:35:55.994251 8567 tablet_loader.cc:96] loaded metadata for tablet 8b90b066ebf3433fb0df9dd90865bc62 (table default.Table_5db9de8011f74b9f9192b39113dfa0c8 [id=dfd09716123a4de7b9fcdd65d1b9ef61])
I20260808 13:35:55.995625 8567 tablet_loader.cc:96] loaded metadata for tablet 8ba098c946794607b2a15bc64098d918 (table default.Table_880132219c91464db6a219d0e98711d5 [id=8a34f9c05f714ace8f24ac7dfd6a8ecf])
I20260808 13:35:56.004776 8567 tablet_loader.cc:96] loaded metadata for tablet 8d06a8cdfabd4adaa4fef064ee6da7d3 (table default.Table_5398341b051f491ca73c85f9c8332eb9 [id=9dbadd28dd50468ba9a99548e008600f])
I20260808 13:35:56.010823 6026 ts_tablet_manager.cc:1428] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.062s user 0.007s sys 0.001s
I20260808 13:35:56.013813 6026 tablet_bootstrap.cc:492] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:56.018260 8929 leader_election.cc:288] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:56.020546 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b8e3245c8ba54f69bc329a5b75115127 (table default.Table_811d61f013fe4c59853421dff8a641bb [id=093c441116e34908b83de704bbb58ee5], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b8e3245c8ba54f69bc329a5b75115127 consensus state has no leader
I20260808 13:35:56.029313 8567 tablet_loader.cc:96] loaded metadata for tablet 8d17615af06b4144a0d7d74c64416451 (table default.Table_f10f80e85d6c4f87b43c1a53b888716c [id=20c27e5e63df49a8bcf6faef31940880])
I20260808 13:35:56.034103 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:56.034971 5630 raft_consensus.cc:3037] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:56.044839 5630 raft_consensus.cc:2454] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:56.036648 8567 tablet_loader.cc:96] loaded metadata for tablet 8e843a211ab24792b9a475cf08e8c49b (table default.Table_8a69b0e5577e4c6f9c76504751d2c1a4 [id=1b8762a3419f47bb9c060294f70d62aa])
I20260808 13:35:56.051798 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09eb729339e47eb8db88eb6509daab4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:56.052937 8567 tablet_loader.cc:96] loaded metadata for tablet 8ec2c581cf62481789171b7a15512fa4 (table default.Table_43f5211ad48d49e982e3ac2625080359 [id=0111d9bdd1f045b197b24461bc37070f])
I20260808 13:35:56.054006 8567 tablet_loader.cc:96] loaded metadata for tablet 8f16ee6c8acb40d994347546fa0fdbe7 (table default.Table_a8ff8993913d4a05a3d8ab3bb339b7ac [id=8ac0944cbe564a949c732ec146a9cf94])
I20260808 13:35:56.042832 8868 leader_election.cc:288] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:56.059239 5771 leader_election.cc:341] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.046674 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09eb729339e47eb8db88eb6509daab4" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:56.057017 6026 tablet_bootstrap.cc:654] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:56.089010 5635 raft_consensus.cc:3037] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:56.093122 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:56.095069 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9e8d48c55349058cf80bf7c2e149e7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:56.099303 5771 leader_election.cc:341] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.100261 8654 raft_consensus.cc:482] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:56.100804 8654 raft_consensus.cc:504] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:56.103415 5771 leader_election.cc:341] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.105226 8654 leader_election.cc:288] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:56.113294 5635 raft_consensus.cc:2454] T d09eb729339e47eb8db88eb6509daab4 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:56.114146 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3e9c6770fd402290b45eed85de6f6f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:56.115010 5379 raft_consensus.cc:2454] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:56.115190 6033 tablet_bootstrap.cc:492] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:56.115615 6033 ts_tablet_manager.cc:1397] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.204s user 0.011s sys 0.001s
I20260808 13:35:56.129387 8567 tablet_loader.cc:96] loaded metadata for tablet 932a328069bd4edf8137f49a5da66a22 (table default.Table_0fdee57de12549eab898e92f30fe691d [id=9ad354d775a84d32bd5eb627356e0a6b])
I20260808 13:35:56.140480 8567 tablet_loader.cc:96] loaded metadata for tablet 953a2a65dea7468c8356931cabb6bfc9 (table default.Table_e101c061f2bf4821b6fe2c6108432735 [id=dbdf4dd6dafe4e5ea1441f7341ec36d7])
I20260808 13:35:56.144919 8567 tablet_loader.cc:96] loaded metadata for tablet 96bed5e6d1aa49a9b14ba86403c985ce (table default.Table_db41a83ae3f34453af64e73302451c19 [id=1816314c549642e3adc9f39e532ab1ca])
I20260808 13:35:56.153194 8954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:35:56.154369 8954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:56.158388 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3e9c6770fd402290b45eed85de6f6f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:56.163437 8567 tablet_loader.cc:96] loaded metadata for tablet 9b427a762b934e988009184bac9ea674 (table default.Table_dbb6f93863704e4e9c573d989a367368 [id=2571402ea4b7478098e9596eb3235370])
I20260808 13:35:56.164659 8567 tablet_loader.cc:96] loaded metadata for tablet 9be987112b0c4fa889d5458c65089098 (table default.Table_0ab0a38241284050a12358d0f8d76032 [id=b713080d065a4fa49a583244a722909c])
I20260808 13:35:56.165956 8567 tablet_loader.cc:96] loaded metadata for tablet 9d20826f93964ba287c7058c07285d64 (table default.Table_ce81522648e14e5daa8c8fb7d55ef7e4 [id=9df6ef937f0044ab9b0ded9876bead2d])
I20260808 13:35:56.167192 8567 tablet_loader.cc:96] loaded metadata for tablet 9dee4b3b14264f3aad0896f629ab5b27 (table default.Table_4cb4f6347fda4d268fd87065c9b063ea [id=11f8a63441444ac1a7aa7165cbe4b696])
I20260808 13:35:56.180557 8567 tablet_loader.cc:96] loaded metadata for tablet 9ef8c23840c848ddb7d8b1023a839e94 (table default.Table_52f5a796f61f4be79a196bc23065491b [id=ee175d53df794e6aaf29faff1fc2bb6f])
I20260808 13:35:56.184614 6046 tablet_bootstrap.cc:492] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:56.185261 6046 ts_tablet_manager.cc:1397] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.321s user 0.017s sys 0.000s
I20260808 13:35:56.190654 8567 tablet_loader.cc:96] loaded metadata for tablet a04d3dd028da4e7f86689e819548d0c5 (table default.Table_f685324c68e647af9888454398dc7d23 [id=01d0f24951d44f7f86fb55decd5990ab])
I20260808 13:35:56.159198 5630 raft_consensus.cc:2454] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:56.202090 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:56.203717 8744 consensus_queue.cc:1035] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:56.119192 5771 leader_election.cc:302] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:56.118579 6033 raft_consensus.cc:348] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:56.147814 5866 raft_consensus.cc:3037] T b8e3245c8ba54f69bc329a5b75115127 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:56.215122 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 96039.5us
I20260808 13:35:56.215224 8944 raft_consensus.cc:2781] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:56.216265 8944 raft_consensus.cc:482] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:56.216744 8944 raft_consensus.cc:3037] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:56.171320 8821 raft_consensus.cc:504] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.221325 8821 leader_election.cc:288] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:56.157634 5772 leader_election.cc:302] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:56.224155 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 66595.1us
I20260808 13:35:56.222319 8567 tablet_loader.cc:96] loaded metadata for tablet a728cc43df834ae19c27d0c1d9b3a52c (table default.Table_4d655ec59e044877882b5cbb4a10a597 [id=716d7d399d82408fb07943a0e0715f88])
I20260808 13:35:56.226402 8567 tablet_loader.cc:96] loaded metadata for tablet a73e98a8b20144508a5a8d451f4bb302 (table default.Table_b339b57a697d413ab7d9ec93220be980 [id=12c0608843fa49cb9c2c931862415447])
I20260808 13:35:56.225773 5772 leader_election.cc:302] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:56.227941 8929 raft_consensus.cc:2781] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:56.228538 8929 raft_consensus.cc:686] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:56.229377 8929 consensus_queue.cc:237] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.227308 8821 raft_consensus.cc:2781] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:56.239306 8821 raft_consensus.cc:686] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:56.240223 8821 consensus_queue.cc:237] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:56.215128 6033 raft_consensus.cc:374] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:56.249316 6033 raft_consensus.cc:729] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:56.249986 6033 consensus_queue.cc:260] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:56.262068 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9e8d48c55349058cf80bf7c2e149e7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:56.262882 5635 raft_consensus.cc:3037] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:56.269156 8567 tablet_loader.cc:96] loaded metadata for tablet a9ea181156f64388ade8a07708b2e814 (table default.Table_577d84019e77413d9ff7373a8315d384 [id=6f200cd5eaff4a63a8b2672962db8c78])
I20260808 13:35:56.272365 6026 tablet_bootstrap.cc:492] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:56.272848 6026 ts_tablet_manager.cc:1397] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.259s user 0.016s sys 0.040s
I20260808 13:35:56.280485 6026 raft_consensus.cc:348] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.281140 6026 raft_consensus.cc:374] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:56.281370 6026 raft_consensus.cc:729] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:56.281921 6026 consensus_queue.cc:260] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.221796 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9e8d48c55349058cf80bf7c2e149e7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:56.211268 6046 raft_consensus.cc:348] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.283522 6046 raft_consensus.cc:374] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:56.283895 6046 raft_consensus.cc:729] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:56.284165 8654 raft_consensus.cc:482] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:35:56.284767 5771 leader_election.cc:341] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.284619 8654 raft_consensus.cc:504] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:56.284611 6046 consensus_queue.cc:260] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.210665 8954 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:56.287487 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37495b29698940edb2924eb612e94bd7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:56.286970 8654 leader_election.cc:288] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:56.288607 5771 leader_election.cc:341] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.292342 6026 ts_tablet_manager.cc:1428] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260808 13:35:56.293282 6026 tablet_bootstrap.cc:492] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:56.297503 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37495b29698940edb2924eb612e94bd7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:56.298120 8567 tablet_loader.cc:96] loaded metadata for tablet ab86016e556c4cdcafeaeda32f56a836 (table default.Table_3961834935a84f0a9646f77db54765ee [id=5115ecc58364490c953100f78c49facf])
I20260808 13:35:56.298192 5637 raft_consensus.cc:2454] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:56.301540 8833 consensus_queue.cc:1035] T b1c41483119a442eb46816aa34f7f319 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:56.308099 8567 tablet_loader.cc:96] loaded metadata for tablet ad151fca07264d9bae6e07ca781c8e04 (table default.Table_ddfcf011ede6442fbc7a45c799fc2cc6 [id=f3bc7b846abd4480b1c1c927c8ad0953])
I20260808 13:35:56.312368 8567 tablet_loader.cc:96] loaded metadata for tablet adf9a03742a14cfe867c578c466838de (table default.Table_fbdbcba42ce04dd68c37a5f32605be9a [id=93919cc7207f4ca09453647840c9bed2])
I20260808 13:35:56.315645 6033 ts_tablet_manager.cc:1428] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.200s user 0.007s sys 0.000s
I20260808 13:35:56.316747 6033 tablet_bootstrap.cc:492] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:56.343815 6046 ts_tablet_manager.cc:1428] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.158s user 0.006s sys 0.000s
I20260808 13:35:56.347620 5772 leader_election.cc:302] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:56.348599 8654 raft_consensus.cc:2781] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:56.349100 8654 raft_consensus.cc:482] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:56.349571 8654 raft_consensus.cc:3037] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:56.375744 6046 tablet_bootstrap.cc:492] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:56.382370 8684 ts_tablet_manager.cc:1918] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:35:56.387440 8944 raft_consensus.cc:504] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:56.392269 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3e9c6770fd402290b45eed85de6f6f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:56.394358 5379 raft_consensus.cc:3037] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:56.401163 8868 raft_consensus.cc:482] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:56.401834 8868 raft_consensus.cc:504] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.408146 5635 raft_consensus.cc:2454] T 1b9e8d48c55349058cf80bf7c2e149e7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:56.408656 8793 raft_consensus.cc:482] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:56.408761 8868 leader_election.cc:288] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:56.420756 6026 tablet_bootstrap.cc:654] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:56.430962 8793 raft_consensus.cc:504] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:56.435664 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007c7222c7bc48329420c740b31ddfae" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:56.435273 8793 leader_election.cc:288] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
W20260808 13:35:56.436415 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 88907us
W20260808 13:35:56.437422 5551 leader_election.cc:341] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.445492 8567 tablet_loader.cc:96] loaded metadata for tablet b0f778e7fa51414cb209fbe1aea92739 (table default.Table_d56fc9f7ea3a43f98fe5a3057b51dbac [id=60103bc8b4444b688fdf2baab858ebf5])
I20260808 13:35:56.445549 8793 raft_consensus.cc:482] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:56.446166 8793 raft_consensus.cc:504] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:56.382802 8684 log.cc:1196] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/4ff163c36c6d42238cce2396102f9a68
I20260808 13:35:56.464704 8684 ts_tablet_manager.cc:1939] T 4ff163c36c6d42238cce2396102f9a68 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:35:56.467741 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:56.469504 5551 leader_election.cc:341] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.474061 8793 leader_election.cc:288] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:56.475124 8567 tablet_loader.cc:96] loaded metadata for tablet b1847d0fecb843cdbc833ad674a503a4 (table default.Table_2d75fe6a74ad43789e515c67b5167ab3 [id=29db8794594e4221be8974258b5127e0])
I20260808 13:35:56.476631 8567 tablet_loader.cc:96] loaded metadata for tablet b1c41483119a442eb46816aa34f7f319 (table default.Table_a16be245c9864b969707b18ee86278d0 [id=8f151f8823464520ae72942a6c79d8bb])
I20260808 13:35:56.494310 6026 tablet_bootstrap.cc:492] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:56.494735 6026 ts_tablet_manager.cc:1397] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.202s user 0.011s sys 0.002s
I20260808 13:35:56.499850 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3e9c6770fd402290b45eed85de6f6f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:56.500550 5635 raft_consensus.cc:3037] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:56.497800 5379 raft_consensus.cc:2454] T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:56.516606 6046 tablet_bootstrap.cc:654] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:56.526022 8567 tablet_loader.cc:96] loaded metadata for tablet b3d489bfe6724b74a830217bfc45a84a (table default.Table_7700dbef158a4ddfa8a3122c18e38817 [id=e6381ed760224b53be85a88376ce8962])
I20260808 13:35:56.502178 6026 raft_consensus.cc:348] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.531112 6026 raft_consensus.cc:374] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:56.531394 6026 raft_consensus.cc:729] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:56.537156 8567 tablet_loader.cc:96] loaded metadata for tablet b49499111fdb45a48e98aa6d0894e2a0 (table default.Table_9d2535df26964377be89e6da0a54e8b4 [id=eb356c7ce7ca4cc1a5a51234f3404e3c])
I20260808 13:35:56.538522 8567 tablet_loader.cc:96] loaded metadata for tablet b6b421ff1f094d8689010b4ed6c3f7af (table default.Table_6eaa73a8f6204e05808ac7d1b397ec04 [id=84e585111c464794ac7ee0e88227ee7a])
I20260808 13:35:56.541456 6026 consensus_queue.cc:260] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.390396 8944 leader_election.cc:288] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:56.560068 8684 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54830 (request call id 485) took 5218 ms. Trace:
W20260808 13:35:56.561883 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 156864us
I20260808 13:35:56.562825 8684 rpcz_store.cc:276] 0808 13:35:51.341456 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.374843 (+ 33387us) service_pool.cc:227] Handling call
0808 13:35:55.890353 (+4515510us) ts_tablet_manager.cc:1145] Deleting tablet 4ff163c36c6d42238cce2396102f9a68
0808 13:35:55.890418 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0808 13:35:56.100455 (+210037us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:56.382123 (+281668us) tablet_metadata.cc:641] Metadata flushed
0808 13:35:56.559970 (+177847us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4514680}
I20260808 13:35:56.564235 8567 tablet_loader.cc:96] loaded metadata for tablet b704d251093c40eba4cc02140046256e (table default.Table_03cf1a01ebc14024a283b7e5ed420074 [id=a45370bd3bbc48279db129789b269480])
I20260808 13:35:56.574672 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba39706cff65404485959865fff1772a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:56.575572 6033 tablet_bootstrap.cc:654] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:56.579162 8567 tablet_loader.cc:96] loaded metadata for tablet b8373a633a07432b9efdf49e932d10f1 (table default.Table_1ab5d21bca4a47978a8f5483f2ba10b2 [id=6d0c2c16340f4cd8805c7692e6275a66])
I20260808 13:35:56.588811 8567 tablet_loader.cc:96] loaded metadata for tablet b8e3245c8ba54f69bc329a5b75115127 (table default.Table_811d61f013fe4c59853421dff8a641bb [id=093c441116e34908b83de704bbb58ee5])
I20260808 13:35:56.590178 8567 tablet_loader.cc:96] loaded metadata for tablet b8ea98afef6841fead641b314ee53a6e (table default.Table_7353a29b611e4f519b3e47a276cd7b5a [id=4694a5eaa9524750b04d6e92ef4720ea])
I20260808 13:35:56.594923 5771 leader_election.cc:302] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:56.597359 8567 tablet_loader.cc:96] loaded metadata for tablet b973f2094a4e420095757820bbb8a767 (table default.Table_187b31571225409e855d0b4783e4b74c [id=0c03629a169f469897599b731ca031e2])
I20260808 13:35:56.621639 8567 tablet_loader.cc:96] loaded metadata for tablet ba6cad4fd8a04f7b87d8e873b8ec8a3f (table default.Table_6b4561dcd8e64216a27618e3599c13d8 [id=fce834aa5aec4694845b2ffc2c9fe962])
I20260808 13:35:56.635979 8567 tablet_loader.cc:96] loaded metadata for tablet bcc0c6ab755c4a6887dabeeef9364201 (table default.Table_a14f7675417240ca88f88da86055bceb [id=16cef8437603476b97a8562edba58a41])
I20260808 13:35:56.637300 8567 tablet_loader.cc:96] loaded metadata for tablet bf780c8e7ec949bd805e7fc9df5f321e (table default.Table_c06c463992cf45db8dacc3018d8bcb8a [id=00bfe57fc39c432690f17b74eab8d1fe])
W20260808 13:35:56.639256 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 44413.4us
W20260808 13:35:56.643280 5771 leader_election.cc:341] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.651901 8967 raft_consensus.cc:2781] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:56.654177 8967 raft_consensus.cc:686] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:56.665426 8967 consensus_queue.cc:237] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:56.675833 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:56.678081 5550 leader_election.cc:341] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.678535 5550 leader_election.cc:302] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:56.680120 8858 raft_consensus.cc:2726] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:56.679780 8978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:35:56.685446 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007c7222c7bc48329420c740b31ddfae" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:56.690160 5550 leader_election.cc:341] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.690564 5550 leader_election.cc:302] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:56.691471 8858 raft_consensus.cc:2726] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:56.683095 8978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:56.699296 5635 raft_consensus.cc:2454] T fc3e9c6770fd402290b45eed85de6f6f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:56.699783 8978 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:35:56.704993 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:35:56.638546 8567 tablet_loader.cc:96] loaded metadata for tablet c1607745896141f68535b751dc211796 (table default.Table_c7297f7910b4405ab0483761e1aed004 [id=b0736fe025cb443ba8cbab989b9a4422])
I20260808 13:35:56.707674 8567 tablet_loader.cc:96] loaded metadata for tablet c2c50fd06b8841c8b5a944401b495d2e (table default.Table_20c0fcbe614b48089d98badce0ef9eb2 [id=4cbe915d8abf43369d194d4087999bc5])
I20260808 13:35:56.708879 8567 tablet_loader.cc:96] loaded metadata for tablet c4c083c8e19d4279af600670f5d1d78d (table default.Table_b938fae0ac404bf6acd3d8bef2b83cfe [id=4965998f372c444d8e2805605dda7532])
I20260808 13:35:56.728997 6026 ts_tablet_manager.cc:1428] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.234s user 0.006s sys 0.000s
I20260808 13:35:56.731133 4819 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 4ff163c36c6d42238cce2396102f9a68 (table default.Table_00b62ebc438b40209bd662e8cd04fb29 [id=e90bcbe52e7b4ad1b60f0324fe158df4]) successfully deleted
I20260808 13:35:56.740212 8567 tablet_loader.cc:96] loaded metadata for tablet c8f1618a1d044898bffbc82107e90840 (table default.Table_7fd510a91deb449091ff7da8c644fae1 [id=1b86c1f240214c88b29348f33a11503d])
I20260808 13:35:56.735898 8858 raft_consensus.cc:482] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:56.757062 8858 raft_consensus.cc:504] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:56.760995 8858 leader_election.cc:288] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:56.765432 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d12106f0d794a79935999a2b8dbe7e9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:56.765396 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d12106f0d794a79935999a2b8dbe7e9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:56.766759 5551 leader_election.cc:341] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:56.769003 5550 leader_election.cc:341] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.769451 5550 leader_election.cc:302] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:56.770529 8793 raft_consensus.cc:482] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:56.771404 8793 raft_consensus.cc:504] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.774643 8793 leader_election.cc:288] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:56.776182 8793 raft_consensus.cc:2726] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:56.779843 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604d2113451642329b877f8beff71618" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:56.781831 8567 tablet_loader.cc:96] loaded metadata for tablet cc4dc66771804e0cb8991da63eacba0c (table default.Table_114cb957447a4943806bdadc6ca86549 [id=000c41391410419bb1d4881b34c602fe])
W20260808 13:35:56.784484 5550 leader_election.cc:341] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.799532 8567 tablet_loader.cc:96] loaded metadata for tablet cc4e6d765eb04ca581db353f93d4f5b5 (table default.Table_533328cbcc13486fb2b3c66451f55ac3 [id=e90cdc01582749b092c452f915c314eb])
I20260808 13:35:56.800868 8567 tablet_loader.cc:96] loaded metadata for tablet ccef7d478a31444481546a15a95e02d9 (table default.Table_9936d69a1f7b4853a7c37d5e8da2077f [id=a7e1be7320c34f3bab091c92ec2c1f0c])
I20260808 13:35:56.802191 8567 tablet_loader.cc:96] loaded metadata for tablet ce39289278cb44cc86d41c3abb516a1d (table default.Table_9758d29f1a0b4ea5bceef3d5ce652ca1 [id=a3d0de3d513c43f29bea23a52883916a])
I20260808 13:35:56.814563 8567 tablet_loader.cc:96] loaded metadata for tablet d09eb729339e47eb8db88eb6509daab4 (table default.Table_233ac5cc38504a5eb9b9cbf426cc19c7 [id=af41016671a644dcaacf0b2602d83672])
I20260808 13:35:56.815924 8567 tablet_loader.cc:96] loaded metadata for tablet d0cc688386f3413ea939dda3a75d6541 (table default.Table_5b856c2026594d708a6d8e38edd5d85f [id=452cc67848cf43efb3d80e76fdca3b9a])
I20260808 13:35:56.817971 8567 tablet_loader.cc:96] loaded metadata for tablet d2be108f26bb48d49216c6b7eef6b9b8 (table default.Table_cab0814763554410aa969d5b417710fb [id=eef293a0bbc94c2db94a64223ef71a5d])
I20260808 13:35:56.842809 8970 raft_consensus.cc:482] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:56.844314 8970 raft_consensus.cc:504] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:56.860918 8567 tablet_loader.cc:96] loaded metadata for tablet d392ae0edc0e4976bed7f3c82145f662 (table default.Table_3e5cfb8683d745f08a7738ed4d2cd13a [id=31e622ce08da4164936a264736f75833])
I20260808 13:35:56.862208 8567 tablet_loader.cc:96] loaded metadata for tablet d3b2cce690cd462aa58ceee2bd28c638 (table default.Table_206fd86bd0f44e8e9dc38631edff49ca [id=1a31b0d0c87b446cb7662a856510b67e])
I20260808 13:35:56.863509 8567 tablet_loader.cc:96] loaded metadata for tablet d3c297fe49c74bceb2cfe2fbece2d0ba (table default.Table_6932f06e7e1e4ec79c76ad6efea73902 [id=665fac81069c47c2b2b787ed500d3a14])
I20260808 13:35:56.753750 8654 raft_consensus.cc:504] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:56.865613 8567 tablet_loader.cc:96] loaded metadata for tablet d4497e1f59c04ad190382259705b1b54 (table default.Table_0913077ace074f11bf8a6222338489c6 [id=e048189f6ef441ce8e17c60f0f5b0cd7])
I20260808 13:35:56.866506 8654 leader_election.cc:288] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:56.867862 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37495b29698940edb2924eb612e94bd7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:56.877254 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 47171.3us
I20260808 13:35:56.803669 6026 tablet_bootstrap.cc:492] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:56.895098 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604d2113451642329b877f8beff71618" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:56.898967 8567 tablet_loader.cc:96] loaded metadata for tablet d5c010e1c8804b7e8e4a58f091cad1d1 (table default.Table_6fc58ab15c1b44c69500f3ad87062bcd [id=562d628bce6b4d96b2013e847c87a8cb])
I20260808 13:35:56.911794 8567 tablet_loader.cc:96] loaded metadata for tablet d9268f5f6dee4112932e106010a3c2be (table default.Table_a450e64d7a494bbcb8f35780fda0ccc8 [id=c4338f1c19ec4210a803a1e2aaaa65d4])
I20260808 13:35:56.914322 8567 tablet_loader.cc:96] loaded metadata for tablet ddd0134819474e7eadd4e3d12d94a7b6 (table default.Table_da0f5516539d4f17b7ad03013cfd9460 [id=b06d9584a664410aad7628dbb2495f85])
W20260808 13:35:56.917481 5551 leader_election.cc:341] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:56.918046 5551 leader_election.cc:302] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:56.904520 6046 tablet_bootstrap.cc:492] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:56.922457 6046 ts_tablet_manager.cc:1397] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.547s user 0.011s sys 0.005s
I20260808 13:35:56.925940 8793 raft_consensus.cc:2726] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:56.869607 5771 leader_election.cc:341] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:56.929953 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 60382.3us
W20260808 13:35:56.788424 8976 log.cc:924] Time spent T b1c41483119a442eb46816aa34f7f319 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Append to log took a long time: real 0.092s user 0.000s sys 0.001s
W20260808 13:35:56.910534 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 380872us
I20260808 13:35:56.919274 6033 tablet_bootstrap.cc:492] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:56.937433 6033 ts_tablet_manager.cc:1397] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.621s user 0.008s sys 0.010s
W20260808 13:35:56.941470 5285 consensus_peers.cc:487] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de6800eceb96475284fdbffb4d23e5e3. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:56.942504 5772 leader_election.cc:302] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:56.951303 8567 tablet_loader.cc:96] loaded metadata for tablet deccbee005534189a274ec40bc91048e (table default.Table_bf5259e68f5c4869bac79b5b0a6a0cd0 [id=acbe4578625746948e76ae16ce884569])
I20260808 13:35:56.952785 8567 tablet_loader.cc:96] loaded metadata for tablet df27489ae6e2456996671e8fa57af566 (table default.Table_702490ba7901458b954c15c4637424b0 [id=1e6b56dff3614e5fb96523262224c209])
W20260808 13:35:56.958101 5772 leader_election.cc:341] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: ba39706cff65404485959865fff1772a
I20260808 13:35:56.974745 8567 tablet_loader.cc:96] loaded metadata for tablet e1d2aae6405349fc887ec8527464e46a (table default.Table_33484efb632a4860bc2485f6713c84d0 [id=26126642d65b4c8bbacd0e4741ed7eee])
I20260808 13:35:57.006240 8567 tablet_loader.cc:96] loaded metadata for tablet eaa049ef916b44458be43ec4e2ccaf1c (table default.Table_f676fb75c0ba45a98625440ed5dd7313 [id=d342034fcd7b45debb1aca898c6b7f68])
I20260808 13:35:57.007153 8970 leader_election.cc:288] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:57.016077 5772 leader_election.cc:302] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:57.009169 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa493b8652e149f988d41bee811f6a2f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:57.017025 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 58949.6us
I20260808 13:35:57.018203 8969 raft_consensus.cc:2726] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:57.021363 5550 leader_election.cc:341] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:57.023383 8946 consensus_queue.cc:1035] T de6800eceb96475284fdbffb4d23e5e3 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:56.958729 8654 raft_consensus.cc:2781] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:57.030709 8654 raft_consensus.cc:686] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:57.031741 8654 consensus_queue.cc:237] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.010236 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa493b8652e149f988d41bee811f6a2f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:57.036260 5551 leader_election.cc:341] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:57.036787 5551 leader_election.cc:302] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:57.037918 8946 raft_consensus.cc:482] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:57.038512 8946 raft_consensus.cc:504] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.041220 8946 leader_election.cc:288] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:57.042712 8793 raft_consensus.cc:2726] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:57.044821 6026 tablet_bootstrap.cc:654] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:57.048679 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:57.049772 5630 raft_consensus.cc:2379] T 538d9a883e9d46668fde30876352d20e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:57.044683 6046 raft_consensus.cc:348] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.059779 6046 raft_consensus.cc:374] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:57.060164 6046 raft_consensus.cc:729] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:57.060964 6046 consensus_queue.cc:260] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.029398 6033 raft_consensus.cc:348] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.068984 6033 raft_consensus.cc:374] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:57.069325 6033 raft_consensus.cc:729] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:57.070007 6033 consensus_queue.cc:260] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.107826 6046 ts_tablet_manager.cc:1428] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.181s user 0.006s sys 0.000s
I20260808 13:35:57.110625 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37495b29698940edb2924eb612e94bd7" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:57.111281 5630 raft_consensus.cc:3037] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:57.116524 8567 tablet_loader.cc:96] loaded metadata for tablet ef2b958e376140f7949aa04d49a4aec6 (table default.Table_82d49330ca914820a6cb8b3591280a90 [id=dc1f8a5aa37a4ad99be0715831a680fd])
I20260808 13:35:57.135864 6033 ts_tablet_manager.cc:1428] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.198s user 0.007s sys 0.000s
I20260808 13:35:57.148376 6046 tablet_bootstrap.cc:492] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:57.153270 6026 tablet_bootstrap.cc:492] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:57.153731 6026 ts_tablet_manager.cc:1397] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.350s user 0.006s sys 0.005s
I20260808 13:35:57.156193 8567 tablet_loader.cc:96] loaded metadata for tablet f26e054d1992452396d3f7334ba3602c (table default.Table_95ed126fb41342c1a6a1539f0a5ebbe3 [id=b301b3d4964b41009ed2903dcbefdfae])
I20260808 13:35:57.163477 8567 tablet_loader.cc:96] loaded metadata for tablet f5f26063029f49d2855428be49bb5a3b (table default.Table_efd769b05e1142088d56528e943072ed [id=31c20a71a6144b238a2da27b7ea21863])
I20260808 13:35:57.164723 8567 tablet_loader.cc:96] loaded metadata for tablet f66b38e83dab4bf1a3d2d1e088a96988 (table default.Table_f1ee5ef8d2d34ba8844275610eb0a394 [id=4296a36ca9b74888aeccfe22e3970de6])
I20260808 13:35:57.165838 8567 tablet_loader.cc:96] loaded metadata for tablet f85b4fd30d6c4903946891a399acf7a0 (table default.Table_f042821be4a8421ea3f64f33a071b22d [id=ea2ba343c17d403d8cb0bf9d80bee453])
I20260808 13:35:57.174763 6033 tablet_bootstrap.cc:492] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:57.189174 6026 raft_consensus.cc:348] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.189878 6026 raft_consensus.cc:374] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:57.190104 6026 raft_consensus.cc:729] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:57.191057 8567 tablet_loader.cc:96] loaded metadata for tablet f935e5a12934443aafee878e0af56b9e (table default.Table_bc6082a17c754033966ecce7c4f9bf26 [id=3a6d52daeb034a88bdcf06112086dff8])
I20260808 13:35:57.192099 8567 tablet_loader.cc:96] loaded metadata for tablet fbe6707a90fd45a2bfda4ec4af1bddd9 (table default.Table_f985e4cebca740fea14f259fcfacec86 [id=e0c4baae657c406b9d7c8c59fb77e10e])
I20260808 13:35:57.193220 8567 tablet_loader.cc:96] loaded metadata for tablet fc16da5c85ff4540ad74d332ca962979 (table default.Table_2269d7b97e6a488b9a86e8240b3b197e [id=7a78c3e7fd434fe48e5d0d8faa195f6e])
I20260808 13:35:57.194294 8567 tablet_loader.cc:96] loaded metadata for tablet fc3e9c6770fd402290b45eed85de6f6f (table default.Table_bf8907064282450eb869d5bb8ec9f1c1 [id=07969203fd174637a09f5b8e83bc7402])
I20260808 13:35:57.212387 8567 tablet_loader.cc:96] loaded metadata for tablet fcf907a9f19f4d64b27be946011a8ddb (table default.Table_86d171579f64458497d484e10e5f630a [id=d99ab852cfb94fb4835c8e9c19d293fc])
I20260808 13:35:57.213559 8567 tablet_loader.cc:96] loaded metadata for tablet fdacfbbaef114ea7a9da62f8a65916f6 (table default.Table_b84003117fd04b17930f7a3ba4e2d05f [id=7f4fa75c41e542928b94ef51cfd2f33a])
I20260808 13:35:57.221685 5630 raft_consensus.cc:2454] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:57.226814 8567 tablet_loader.cc:96] loaded metadata for tablet ffbdf03ebe4e48a98450e90a3a78181c (table default.Table_510c442cafba44a9bf48a292ca01fbd4 [id=212e1314b3c04c4d8dd06d5730fa25cf])
I20260808 13:35:57.236565 6046 tablet_bootstrap.cc:654] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:57.236657 8969 raft_consensus.cc:482] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:57.237088 8969 raft_consensus.cc:504] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.239166 8969 leader_election.cc:288] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:57.240587 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:57.241200 5383 raft_consensus.cc:2454] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 2.
I20260808 13:35:57.242008 5771 leader_election.cc:302] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:57.242889 8654 raft_consensus.cc:2781] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 13:35:57.243341 8654 raft_consensus.cc:482] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:57.243716 8654 raft_consensus.cc:3037] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:35:57.250255 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538d9a883e9d46668fde30876352d20e" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:57.254254 8567 catalog_manager.cc:1476] Time spent T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: Loading table and tablet metadata into memory: real 7.637s user 0.710s sys 0.101s
I20260808 13:35:57.260269 8793 raft_consensus.cc:482] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:57.260917 8793 raft_consensus.cc:504] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.263911 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342027968b884f1cbc78b1371318dfc9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:57.265388 5551 leader_election.cc:341] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:57.268653 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342027968b884f1cbc78b1371318dfc9" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:57.270423 5550 leader_election.cc:341] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:57.270946 5550 leader_election.cc:302] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:57.273403 8970 raft_consensus.cc:2726] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:57.276417 8946 raft_consensus.cc:482] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:57.256667 4977 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 4 LEADER]: Rejecting Update request from peer 3e672d7386144b299688abea9c64854c for earlier term 3. Current term is 4. Ops: []
I20260808 13:35:57.277734 8567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:35:57.278234 4977 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60280 (request call id 854) took 7368 ms. Trace:
I20260808 13:35:57.277019 8946 raft_consensus.cc:504] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.257207 4978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:57.257915 4967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:57.258329 4973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:57.259408 4971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
W20260808 13:35:57.284061 4967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34412 (request call id 11) took 1074 ms (client timeout 727 ms). Trace:
W20260808 13:35:57.284555 4971 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60280 (request call id 858) took 3044 ms (client timeout 742 ms). Trace:
W20260808 13:35:57.284507 4967 rpcz_store.cc:269] 0808 13:35:56.210010 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:56.210226 (+ 216us) service_pool.cc:227] Handling call
0808 13:35:57.283961 (+1073735us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260808 13:35:57.284890 4971 rpcz_store.cc:269] 0808 13:35:54.240511 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:54.240682 (+ 171us) service_pool.cc:227] Handling call
0808 13:35:57.284481 (+3043799us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:57.285826 4971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
W20260808 13:35:57.286633 4971 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60280 (request call id 859) took 587 ms (client timeout 749 ms). Trace:
I20260808 13:35:57.286783 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37495b29698940edb2924eb612e94bd7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:57.286957 4971 rpcz_store.cc:269] 0808 13:35:56.699102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:57.285622 (+586520us) service_pool.cc:227] Handling call
0808 13:35:57.286542 (+ 920us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:57.287753 5635 raft_consensus.cc:2379] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:57.287791 4971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:57.260602 4976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
W20260808 13:35:57.289067 4976 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60280 (request call id 856) took 5241 ms (client timeout 728 ms). Trace:
I20260808 13:35:57.289180 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
W20260808 13:35:57.289435 4976 rpcz_store.cc:269] 0808 13:35:52.047375 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:52.047584 (+ 209us) service_pool.cc:227] Handling call
0808 13:35:57.289001 (+5241417us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:57.261066 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
W20260808 13:35:57.290505 4970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34412 (request call id 10) took 2506 ms (client timeout 602 ms). Trace:
I20260808 13:35:57.291045 4820 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
W20260808 13:35:57.290853 4970 rpcz_store.cc:269] 0808 13:35:54.784263 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:54.784918 (+ 655us) service_pool.cc:227] Handling call
0808 13:35:57.290438 (+2505520us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:57.261729 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
W20260808 13:35:57.292171 4974 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34412 (request call id 8) took 4657 ms (client timeout 618 ms). Trace:
W20260808 13:35:57.292582 4974 rpcz_store.cc:269] 0808 13:35:52.634987 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:52.635173 (+ 186us) service_pool.cc:227] Handling call
0808 13:35:57.292067 (+4656894us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:57.262413 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
W20260808 13:35:57.293898 4972 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34412 (request call id 9) took 3201 ms (client timeout 602 ms). Trace:
W20260808 13:35:57.294325 4972 rpcz_store.cc:269] 0808 13:35:54.092529 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:54.092768 (+ 239us) service_pool.cc:227] Handling call
0808 13:35:57.293815 (+3201047us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:57.263036 4980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
W20260808 13:35:57.295581 4980 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60280 (request call id 855) took 6189 ms (client timeout 728 ms). Trace:
W20260808 13:35:57.295936 4980 rpcz_store.cc:269] 0808 13:35:51.106494 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.106654 (+ 160us) service_pool.cc:227] Handling call
0808 13:35:57.295439 (+6188785us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260808 13:35:57.284083 4973 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60280 (request call id 857) took 3832 ms (client timeout 742 ms). Trace:
W20260808 13:35:57.298544 4973 rpcz_store.cc:269] 0808 13:35:53.451250 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:53.451463 (+ 213us) service_pool.cc:227] Handling call
0808 13:35:57.283974 (+3832511us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260808 13:35:57.282562 4978 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34412 (request call id 7) took 5329 ms (client timeout 618 ms). Trace:
W20260808 13:35:57.299325 4978 rpcz_store.cc:269] 0808 13:35:51.952975 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.955152 (+ 2177us) service_pool.cc:227] Handling call
0808 13:35:57.282439 (+5327287us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:57.278587 4977 rpcz_store.cc:276] 0808 13:35:49.909978 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:49.910102 (+ 124us) service_pool.cc:227] Handling call
0808 13:35:57.256312 (+7346210us) raft_consensus.cc:1457] Updating replica for 0 ops
0808 13:35:57.277928 (+ 21616us) raft_consensus.cc:1683] Filling consensus response to leader.
0808 13:35:57.278157 (+ 229us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:35:57.300848 8793 leader_election.cc:288] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:57.301070 9003 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:35:57.307065 4961 rpcz_store.cc:275] Call kudu.master.MasterService.ConnectToMaster from 127.0.0.1:49444 (request call id 1) took 5957 ms. Trace:
I20260808 13:35:57.307453 4961 rpcz_store.cc:276] 0808 13:35:51.344157 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:51.344278 (+ 121us) service_pool.cc:227] Handling call
0808 13:35:57.281621 (+5937343us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b54000118d4. stack: 00007f04bb18e033 00007f04bb18df30 00007f04bad0f534 00000000004db2b4 00000000004db249 00007f04c209a471 00007f04c211dfe9 00007f04c40a8bd6 00007f04c40e6f87 00007f04c1c1d534 00007f04c1c1d4c2 00007f04c1c1d3f1 00007f04c1c1d36c 00007f04c1c1c622 00007f04bfec494c 00007f04bfec3d86
0808 13:35:57.301482 (+ 19861us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":47490304}
I20260808 13:35:57.313355 8970 raft_consensus.cc:482] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 98dd96f425fc4ab3a6a54f2dfaaf9fab)
I20260808 13:35:57.307947 8946 leader_election.cc:288] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:57.313853 8970 raft_consensus.cc:504] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.315973 8970 leader_election.cc:288] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:57.324340 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:57.325520 5379 raft_consensus.cc:2379] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb in current term 2: Already voted for candidate 946c9d8462644deba5b45fa724f1923c in this term.
I20260808 13:35:57.271260 6026 consensus_queue.cc:260] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.350893 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:57.361253 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:57.362069 5635 raft_consensus.cc:2454] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:57.312709 8654 raft_consensus.cc:504] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.364351 5869 raft_consensus.cc:2354] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb for earlier term 2. Current term is 3.
I20260808 13:35:57.364650 8654 leader_election.cc:288] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 3 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:57.365199 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:57.365708 5383 raft_consensus.cc:3037] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:35:57.366505 5550 leader_election.cc:398] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 2 pre-election: Vote denied by peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) with higher term. Message: Invalid argument: T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb for earlier term 2. Current term is 3.
I20260808 13:35:57.367125 5550 leader_election.cc:401] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260808 13:35:57.368218 8970 raft_consensus.cc:3037] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Advancing to term 3
I20260808 13:35:57.379463 8970 raft_consensus.cc:2726] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) with higher term. Message: Invalid argument: T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cd987c3dbdb4168beb73cd7e577c7cb for earlier term 2. Current term is 3.
I20260808 13:35:57.382710 5383 raft_consensus.cc:2454] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 3.
I20260808 13:35:57.383752 5771 leader_election.cc:302] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:57.384655 8929 raft_consensus.cc:2781] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 3 FOLLOWER]: Leader election won for term 3
I20260808 13:35:57.385113 8929 raft_consensus.cc:686] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 3 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:57.386101 8929 consensus_queue.cc:237] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.394701 5772 leader_election.cc:302] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:57.395498 8929 raft_consensus.cc:2781] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
W20260808 13:35:57.400959 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 80813.2us
W20260808 13:35:57.401857 7535 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 112769us
I20260808 13:35:57.406639 5283 leader_election.cc:302] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:57.407184 6033 tablet_bootstrap.cc:654] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:57.407348 8833 raft_consensus.cc:3037] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:57.411517 6026 ts_tablet_manager.cc:1428] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.258s user 0.007s sys 0.000s
I20260808 13:35:57.394289 9002 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:35:57.417456 6026 tablet_bootstrap.cc:492] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:57.418831 8969 consensus_queue.cc:1035] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:57.419651 8929 raft_consensus.cc:686] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:57.420495 8929 consensus_queue.cc:237] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.425462 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37495b29698940edb2924eb612e94bd7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:57.450390 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1de317c48eaa43dca038410840483f52" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:57.464884 5637 raft_consensus.cc:2454] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 3.
W20260808 13:35:57.477684 5771 consensus_peers.cc:487] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:57.481418 8929 raft_consensus.cc:482] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:57.482002 8929 raft_consensus.cc:504] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.484710 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d68ab73bae4dd9af46e907b1fd198c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:57.484797 8929 leader_election.cc:288] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:57.486071 5771 leader_election.cc:341] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:57.471367 8833 raft_consensus.cc:2726] T 538d9a883e9d46668fde30876352d20e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:57.486934 6046 tablet_bootstrap.cc:492] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:57.487357 6046 ts_tablet_manager.cc:1397] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.378s user 0.009s sys 0.002s
I20260808 13:35:57.490520 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d68ab73bae4dd9af46e907b1fd198c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:57.491219 5636 raft_consensus.cc:2454] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:57.497407 5772 leader_election.cc:302] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:57.498102 8868 raft_consensus.cc:2781] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:57.498440 8868 raft_consensus.cc:482] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:57.498777 8868 raft_consensus.cc:3037] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:57.498662 8969 consensus_queue.cc:1035] T d09eb729339e47eb8db88eb6509daab4 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:35:57.499267 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 46651us
I20260808 13:35:57.499950 5283 leader_election.cc:302] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:57.501248 8833 raft_consensus.cc:3037] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:57.523865 8929 consensus_queue.cc:1035] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:57.545279 8868 raft_consensus.cc:504] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.548588 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d68ab73bae4dd9af46e907b1fd198c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:57.549204 8833 raft_consensus.cc:2726] T 37495b29698940edb2924eb612e94bd7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:57.550069 5771 leader_election.cc:341] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:57.550997 6033 tablet_bootstrap.cc:492] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:57.584734 8567 catalog_manager.cc:1259] Loaded cluster ID: 9d0f658ea6c34f378fe8b1ad811bf2bf
I20260808 13:35:57.585042 8567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:35:57.589811 8969 consensus_queue.cc:1035] T 538d9a883e9d46668fde30876352d20e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:57.602269 6033 ts_tablet_manager.cc:1397] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.428s user 0.009s sys 0.003s
I20260808 13:35:57.624349 6046 raft_consensus.cc:348] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.625196 6046 raft_consensus.cc:374] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:57.625543 6046 raft_consensus.cc:729] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:57.628113 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d68ab73bae4dd9af46e907b1fd198c" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:57.628634 5637 raft_consensus.cc:3037] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:57.634719 8868 leader_election.cc:288] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:57.638784 5637 raft_consensus.cc:2454] T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:57.639983 5772 leader_election.cc:302] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
W20260808 13:35:57.642840 9019 log.cc:924] Time spent T fc3e9c6770fd402290b45eed85de6f6f P 946c9d8462644deba5b45fa724f1923c: Append to log took a long time: real 0.067s user 0.001s sys 0.000s
I20260808 13:35:57.645308 8969 raft_consensus.cc:2781] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:57.647487 8969 raft_consensus.cc:686] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:57.648478 8969 consensus_queue.cc:237] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.654831 6026 tablet_bootstrap.cc:654] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:57.672475 6033 raft_consensus.cc:348] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.673444 6033 raft_consensus.cc:374] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:57.673787 6033 raft_consensus.cc:729] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:57.674517 6033 consensus_queue.cc:260] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.626359 6046 consensus_queue.cc:260] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.702702 6046 ts_tablet_manager.cc:1428] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.143s user 0.006s sys 0.000s
I20260808 13:35:57.677583 8654 consensus_queue.cc:1035] T fc3e9c6770fd402290b45eed85de6f6f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:57.713230 6033 ts_tablet_manager.cc:1428] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.106s user 0.006s sys 0.000s
I20260808 13:35:57.714241 6033 tablet_bootstrap.cc:492] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:57.717024 6026 tablet_bootstrap.cc:492] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:57.717577 6026 ts_tablet_manager.cc:1397] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.300s user 0.005s sys 0.006s
I20260808 13:35:57.720429 6026 raft_consensus.cc:348] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:57.738173 6046 tablet_bootstrap.cc:492] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:57.789776 9029 raft_consensus.cc:482] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:57.790258 9029 raft_consensus.cc:504] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.792723 9029 leader_election.cc:288] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:57.793092 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356e15bbcd26409a9d6302599816e15f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:57.796270 5771 leader_election.cc:341] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:57.829593 6033 tablet_bootstrap.cc:654] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:57.794785 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356e15bbcd26409a9d6302599816e15f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:57.832582 5630 raft_consensus.cc:2454] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:57.833851 5772 leader_election.cc:302] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:57.835246 6026 raft_consensus.cc:374] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:57.836915 6046 tablet_bootstrap.cc:654] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:57.837911 6026 raft_consensus.cc:729] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
W20260808 13:35:57.838681 7863 debug-util.cc:398] Leaking SignalData structure 0x7b08000e50a0 after lost signal to thread 6785
W20260808 13:35:57.839365 7863 debug-util.cc:398] Leaking SignalData structure 0x7b08000d1b00 after lost signal to thread 7879
I20260808 13:35:57.838764 6026 consensus_queue.cc:260] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:57.843621 8929 raft_consensus.cc:2781] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:57.843954 8929 raft_consensus.cc:482] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:57.844292 8929 raft_consensus.cc:3037] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:57.852349 8929 raft_consensus.cc:504] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.854527 8929 leader_election.cc:288] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:57.857504 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356e15bbcd26409a9d6302599816e15f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:57.858556 4821 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0d0018c5333b4fde8ad4ce874660eb97 (table default.Table_db29cf6251b947269c7a48cfc05ea2e5 [id=86e7a76979764ac09c964217ebe7fae4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0d0018c5333b4fde8ad4ce874660eb97 consensus state has no leader
W20260808 13:35:57.858780 5771 leader_election.cc:341] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:57.855589 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356e15bbcd26409a9d6302599816e15f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:57.873875 5634 raft_consensus.cc:3037] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:57.882841 6026 ts_tablet_manager.cc:1428] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.165s user 0.007s sys 0.000s
I20260808 13:35:57.903923 6033 tablet_bootstrap.cc:492] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:57.904407 6033 ts_tablet_manager.cc:1397] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.190s user 0.010s sys 0.000s
I20260808 13:35:57.907490 6033 raft_consensus.cc:348] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.908123 6033 raft_consensus.cc:374] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:57.908375 6033 raft_consensus.cc:729] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:57.918011 5634 raft_consensus.cc:2454] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:57.919248 5772 leader_election.cc:302] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:57.920616 8868 raft_consensus.cc:2781] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:57.921392 8868 raft_consensus.cc:686] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:57.922755 6046 tablet_bootstrap.cc:492] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:57.923273 6046 ts_tablet_manager.cc:1397] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.185s user 0.009s sys 0.002s
I20260808 13:35:57.922909 8868 consensus_queue.cc:237] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.930886 6033 consensus_queue.cc:260] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.933494 6033 ts_tablet_manager.cc:1428] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.029s user 0.004s sys 0.002s
I20260808 13:35:57.934422 6033 tablet_bootstrap.cc:492] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:57.934947 6046 raft_consensus.cc:348] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.935681 6046 raft_consensus.cc:374] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:57.935957 6046 raft_consensus.cc:729] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:57.936509 6046 consensus_queue.cc:260] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:57.941917 8970 raft_consensus.cc:482] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:57.942375 8970 raft_consensus.cc:504] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:57.949714 8970 leader_election.cc:288] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:57.950786 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b314734698ed423ba9cf4134db5846ec" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:57.952520 5550 leader_election.cc:341] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:57.992343 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b314734698ed423ba9cf4134db5846ec" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:57.995452 5551 leader_election.cc:341] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:57.995904 5551 leader_election.cc:302] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:57.998556 8970 raft_consensus.cc:2726] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:57.999251 5771 consensus_peers.cc:487] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:58.010063 6046 ts_tablet_manager.cc:1428] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.087s user 0.008s sys 0.000s
I20260808 13:35:58.016428 6046 tablet_bootstrap.cc:492] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:58.027798 6033 tablet_bootstrap.cc:654] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.030795 6026 tablet_bootstrap.cc:492] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:58.031294 9038 raft_consensus.cc:482] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.031900 9038 raft_consensus.cc:504] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.037295 9038 leader_election.cc:288] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:58.038298 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7f6487e28f4aaa94d448be71f8a3c2" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:58.040791 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7f6487e28f4aaa94d448be71f8a3c2" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:58.042349 5550 leader_election.cc:341] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:58.043112 5551 leader_election.cc:341] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.043460 5551 leader_election.cc:302] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:58.044792 8970 raft_consensus.cc:2726] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:58.045709 8868 consensus_queue.cc:1035] T 1b9e8d48c55349058cf80bf7c2e149e7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:58.051232 6046 tablet_bootstrap.cc:654] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.066210 6046 tablet_bootstrap.cc:492] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:58.067131 6046 ts_tablet_manager.cc:1397] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.051s user 0.006s sys 0.016s
I20260808 13:35:58.067220 9038 raft_consensus.cc:482] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.067680 9038 raft_consensus.cc:504] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.069590 6046 raft_consensus.cc:348] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.070405 6046 raft_consensus.cc:374] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.070737 6046 raft_consensus.cc:729] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.071461 6046 consensus_queue.cc:260] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.072275 9038 leader_election.cc:288] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:58.075052 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f778e7fa51414cb209fbe1aea92739" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:58.076364 5551 leader_election.cc:341] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.077267 6026 tablet_bootstrap.cc:654] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.080421 6046 ts_tablet_manager.cc:1428] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.013s user 0.010s sys 0.001s
I20260808 13:35:58.090685 6046 tablet_bootstrap.cc:492] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:58.103137 6046 tablet_bootstrap.cc:654] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.124785 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f778e7fa51414cb209fbe1aea92739" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:58.127977 5550 leader_election.cc:341] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.128432 5550 leader_election.cc:302] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:58.129623 9040 raft_consensus.cc:2726] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:58.140785 6026 tablet_bootstrap.cc:492] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:58.160621 6026 ts_tablet_manager.cc:1397] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.130s user 0.016s sys 0.001s
I20260808 13:35:58.175812 6046 tablet_bootstrap.cc:492] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:58.176402 6046 ts_tablet_manager.cc:1397] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.088s user 0.009s sys 0.003s
I20260808 13:35:58.178846 6046 raft_consensus.cc:348] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.179665 6046 raft_consensus.cc:374] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.179963 6046 raft_consensus.cc:729] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.180661 6046 consensus_queue.cc:260] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.181818 6026 raft_consensus.cc:348] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:58.184883 6046 ts_tablet_manager.cc:1428] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260808 13:35:58.185228 6026 raft_consensus.cc:374] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.185491 6033 tablet_bootstrap.cc:492] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:58.185911 6033 ts_tablet_manager.cc:1397] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.252s user 0.015s sys 0.006s
I20260808 13:35:58.186185 6026 raft_consensus.cc:729] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.188174 6033 raft_consensus.cc:348] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.188802 6033 raft_consensus.cc:374] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.189042 6033 raft_consensus.cc:729] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.189762 6033 consensus_queue.cc:260] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.190963 6026 consensus_queue.cc:260] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:58.198494 6026 ts_tablet_manager.cc:1428] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20260808 13:35:58.204835 5634 raft_consensus.cc:1261] T 1de317c48eaa43dca038410840483f52 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 3 FOLLOWER]: Refusing update from remote peer 98dd96f425fc4ab3a6a54f2dfaaf9fab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260808 13:35:58.206208 8868 consensus_queue.cc:1035] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:58.209499 6046 tablet_bootstrap.cc:492] T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:58.231256 6046 tablet_bootstrap.cc:654] T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.221822 6033 ts_tablet_manager.cc:1428] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.036s user 0.002s sys 0.004s
I20260808 13:35:58.239910 6026 tablet_bootstrap.cc:492] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:58.266806 8929 raft_consensus.cc:482] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.267455 8929 raft_consensus.cc:504] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.269690 6033 tablet_bootstrap.cc:492] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:58.271718 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3272708a835c432c878cc330e44490c5" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:58.272553 5634 raft_consensus.cc:2454] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:58.278363 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3272708a835c432c878cc330e44490c5" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:58.280670 5771 leader_election.cc:341] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.270465 8929 leader_election.cc:288] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:58.307439 6046 tablet_bootstrap.cc:492] T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:58.307945 6046 ts_tablet_manager.cc:1397] T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.099s user 0.010s sys 0.000s
I20260808 13:35:58.307727 5772 leader_election.cc:302] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:58.311370 6046 raft_consensus.cc:348] T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.312134 6046 raft_consensus.cc:374] T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.312451 6046 raft_consensus.cc:729] T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.313364 6046 consensus_queue.cc:260] T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.321331 8929 raft_consensus.cc:2781] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:58.321843 8929 raft_consensus.cc:482] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:58.322317 8929 raft_consensus.cc:3037] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.323745 6046 ts_tablet_manager.cc:1428] T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260808 13:35:58.324241 6033 tablet_bootstrap.cc:654] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.333029 9040 raft_consensus.cc:482] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.333874 9040 raft_consensus.cc:504] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.336738 9040 leader_election.cc:288] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:58.337574 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01ca2e4d6a2a4e2bb2dd63af8c125e9f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:58.338212 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01ca2e4d6a2a4e2bb2dd63af8c125e9f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:58.339676 5551 leader_election.cc:341] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:58.340687 5550 leader_election.cc:341] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.341086 5550 leader_election.cc:302] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:58.342276 9040 raft_consensus.cc:2726] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:58.350406 5379 raft_consensus.cc:1261] T 1de317c48eaa43dca038410840483f52 P 946c9d8462644deba5b45fa724f1923c [term 3 FOLLOWER]: Refusing update from remote peer 98dd96f425fc4ab3a6a54f2dfaaf9fab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260808 13:35:58.350688 6046 tablet_bootstrap.cc:492] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:58.351923 8969 consensus_queue.cc:1035] T 1de317c48eaa43dca038410840483f52 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:58.366525 9040 raft_consensus.cc:482] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.367205 9040 raft_consensus.cc:504] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.372329 9040 leader_election.cc:288] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:58.372539 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7e8060cab44121acf6dadd2c2873b3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:58.374251 5551 leader_election.cc:341] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.378464 6046 tablet_bootstrap.cc:654] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.378773 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7e8060cab44121acf6dadd2c2873b3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:58.383895 5550 leader_election.cc:341] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.384338 5550 leader_election.cc:302] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:58.394853 9040 raft_consensus.cc:2726] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:58.398787 9053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:35:58.399510 9053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:58.403769 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:58.451076 6026 tablet_bootstrap.cc:654] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.455896 9053 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:35:58.458315 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:35:58.466015 4823 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:35:58.467482 8567 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:35:58.470042 9053 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:35:58.478634 8833 raft_consensus.cc:482] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.479375 8833 raft_consensus.cc:504] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.482084 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9e8d48c55349058cf80bf7c2e149e7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
W20260808 13:35:58.482506 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 70494.2us
I20260808 13:35:58.484127 8833 leader_election.cc:288] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:58.486452 6046 tablet_bootstrap.cc:492] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:58.487020 6046 ts_tablet_manager.cc:1397] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.148s user 0.012s sys 0.004s
I20260808 13:35:58.491291 9013 raft_consensus.cc:482] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.491679 9013 raft_consensus.cc:504] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.495306 9013 leader_election.cc:288] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:58.495872 5285 leader_election.cc:341] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: ba39706cff65404485959865fff1772a
I20260808 13:35:58.497159 9013 raft_consensus.cc:482] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.497865 9013 raft_consensus.cc:504] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.499701 6033 tablet_bootstrap.cc:492] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:58.500119 6033 ts_tablet_manager.cc:1397] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.231s user 0.012s sys 0.013s
I20260808 13:35:58.501338 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea8f5322a404837b91e1d96749926b7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:58.501906 5630 raft_consensus.cc:2454] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:58.502961 5285 leader_election.cc:302] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:58.503662 8833 raft_consensus.cc:2781] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:58.503760 9013 leader_election.cc:288] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:58.504002 8833 raft_consensus.cc:482] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:58.504395 8833 raft_consensus.cc:3037] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.513752 8833 raft_consensus.cc:504] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.489535 6046 raft_consensus.cc:348] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.519161 6046 raft_consensus.cc:374] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.519536 6046 raft_consensus.cc:729] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.523602 8929 raft_consensus.cc:504] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.527176 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba39706cff65404485959865fff1772a" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:58.527791 5869 raft_consensus.cc:2454] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:58.524981 6046 consensus_queue.cc:260] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.528795 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea8f5322a404837b91e1d96749926b7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:58.529178 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9e8d48c55349058cf80bf7c2e149e7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:58.529369 5875 raft_consensus.cc:2454] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:58.538718 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea8f5322a404837b91e1d96749926b7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:58.539263 5630 raft_consensus.cc:3037] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.539629 8929 leader_election.cc:288] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:58.540970 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3272708a835c432c878cc330e44490c5" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
W20260808 13:35:58.542651 5771 leader_election.cc:341] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.545369 6033 raft_consensus.cc:348] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.546070 6033 raft_consensus.cc:374] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.546350 6033 raft_consensus.cc:729] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.546991 6033 consensus_queue.cc:260] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.546429 8833 leader_election.cc:288] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:58.552625 9013 raft_consensus.cc:482] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.552997 5630 raft_consensus.cc:2454] T 9ea8f5322a404837b91e1d96749926b7 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:58.553072 9013 raft_consensus.cc:504] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.554062 5285 leader_election.cc:302] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:58.555382 8744 raft_consensus.cc:2781] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:58.555487 9013 leader_election.cc:288] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:58.555841 8744 raft_consensus.cc:686] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:58.556699 8744 consensus_queue.cc:237] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.559418 5283 leader_election.cc:302] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:58.560312 9043 raft_consensus.cc:2781] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:58.560637 9043 raft_consensus.cc:482] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:58.560978 9043 raft_consensus.cc:3037] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.561718 5283 leader_election.cc:302] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:58.563689 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3272708a835c432c878cc330e44490c5" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:58.564230 5637 raft_consensus.cc:3037] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.564899 9013 raft_consensus.cc:2726] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:58.565655 9013 raft_consensus.cc:482] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.566022 9013 raft_consensus.cc:504] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.568439 9013 leader_election.cc:288] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:58.569118 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09eb729339e47eb8db88eb6509daab4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:58.569104 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356e15bbcd26409a9d6302599816e15f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:58.571873 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356e15bbcd26409a9d6302599816e15f" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:58.572162 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea8f5322a404837b91e1d96749926b7" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:58.571475 5634 raft_consensus.cc:2379] T 356e15bbcd26409a9d6302599816e15f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:58.572645 5869 raft_consensus.cc:3037] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.574049 5637 raft_consensus.cc:2454] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:58.574337 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09eb729339e47eb8db88eb6509daab4" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:58.574975 9043 raft_consensus.cc:504] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.580329 6026 tablet_bootstrap.cc:492] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:58.582023 6026 ts_tablet_manager.cc:1397] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.348s user 0.006s sys 0.011s
I20260808 13:35:58.583122 5378 raft_consensus.cc:3037] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:35:58.584350 5285 leader_election.cc:341] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: ba39706cff65404485959865fff1772a
I20260808 13:35:58.585637 6046 ts_tablet_manager.cc:1428] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.098s user 0.004s sys 0.001s
I20260808 13:35:58.585515 6026 raft_consensus.cc:348] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:58.586153 6026 raft_consensus.cc:374] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.586375 6026 raft_consensus.cc:729] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.586539 6046 tablet_bootstrap.cc:492] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:58.591369 9043 leader_election.cc:288] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:58.592586 5283 leader_election.cc:302] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:58.594246 5283 leader_election.cc:302] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:58.596493 8744 raft_consensus.cc:3037] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.604342 5772 leader_election.cc:302] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:58.604439 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba39706cff65404485959865fff1772a" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:58.606159 5875 raft_consensus.cc:3037] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.607226 6033 ts_tablet_manager.cc:1428] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.107s user 0.007s sys 0.000s
I20260808 13:35:58.608091 6033 tablet_bootstrap.cc:492] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:58.612377 6026 consensus_queue.cc:260] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:58.612860 5875 raft_consensus.cc:2454] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:58.615378 6026 ts_tablet_manager.cc:1428] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.033s user 0.005s sys 0.000s
I20260808 13:35:58.620143 5869 raft_consensus.cc:2454] T 9ea8f5322a404837b91e1d96749926b7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:58.627909 8744 raft_consensus.cc:2726] T 356e15bbcd26409a9d6302599816e15f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:58.639765 6046 tablet_bootstrap.cc:654] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.639894 6026 tablet_bootstrap.cc:492] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:58.642733 9013 raft_consensus.cc:2726] T d09eb729339e47eb8db88eb6509daab4 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:58.645545 6026 tablet_bootstrap.cc:654] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.649246 9052 raft_consensus.cc:2781] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:58.649802 9052 raft_consensus.cc:686] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:58.650727 9052 consensus_queue.cc:237] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.653508 9067 raft_consensus.cc:482] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.653916 9067 raft_consensus.cc:504] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:58.654292 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 50024.7us
I20260808 13:35:58.655802 9067 leader_election.cc:288] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:58.656622 5283 leader_election.cc:302] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb
I20260808 13:35:58.659773 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20113491b108476b95b8f32b69176d0e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:58.661048 5771 leader_election.cc:341] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.658203 9043 raft_consensus.cc:2781] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:58.665465 9065 consensus_queue.cc:1035] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:58.666221 9043 raft_consensus.cc:686] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:58.667068 9043 consensus_queue.cc:237] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.679698 9040 raft_consensus.cc:482] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.680163 9040 raft_consensus.cc:504] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:58.687605 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293e9540498e420d8ec873472d0d29fc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:58.689312 6046 tablet_bootstrap.cc:492] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:58.689880 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293e9540498e420d8ec873472d0d29fc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:58.690891 5550 leader_election.cc:341] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.686591 9040 leader_election.cc:288] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
W20260808 13:35:58.692788 5551 leader_election.cc:341] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.693192 5551 leader_election.cc:302] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:58.694382 9040 raft_consensus.cc:2726] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:58.704107 6026 tablet_bootstrap.cc:492] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:58.704598 6026 ts_tablet_manager.cc:1397] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.065s user 0.006s sys 0.003s
I20260808 13:35:58.726416 9067 consensus_queue.cc:1035] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:58.728295 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20113491b108476b95b8f32b69176d0e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:58.728892 5630 raft_consensus.cc:2454] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:58.729880 6046 ts_tablet_manager.cc:1397] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.143s user 0.011s sys 0.000s
I20260808 13:35:58.732206 6046 raft_consensus.cc:348] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.732846 6046 raft_consensus.cc:374] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.733220 6046 raft_consensus.cc:729] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.733785 6046 consensus_queue.cc:260] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.741437 6033 tablet_bootstrap.cc:654] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.819283 6026 raft_consensus.cc:348] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.819993 6026 raft_consensus.cc:374] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.820271 6026 raft_consensus.cc:729] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.821285 5772 leader_election.cc:302] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:58.833402 5379 raft_consensus.cc:3037] T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.839016 6026 consensus_queue.cc:260] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.844468 6046 ts_tablet_manager.cc:1428] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.114s user 0.006s sys 0.000s
I20260808 13:35:58.861475 6026 ts_tablet_manager.cc:1428] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.157s user 0.004s sys 0.003s
I20260808 13:35:58.863324 6046 tablet_bootstrap.cc:492] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:35:58.870121 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 48920.4us
I20260808 13:35:58.872083 9029 consensus_queue.cc:1035] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:58.871114 9052 raft_consensus.cc:2781] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:58.874138 9052 raft_consensus.cc:482] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:58.874472 9052 raft_consensus.cc:3037] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.908602 6033 tablet_bootstrap.cc:492] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:58.909274 6033 ts_tablet_manager.cc:1397] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.301s user 0.006s sys 0.009s
I20260808 13:35:58.916639 9052 raft_consensus.cc:504] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.954124 9040 raft_consensus.cc:482] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:58.954811 9040 raft_consensus.cc:504] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:58.958076 9040 leader_election.cc:288] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:58.958927 6026 tablet_bootstrap.cc:492] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:35:58.959234 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 114670us
I20260808 13:35:58.967978 9101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:35:58.968299 6046 tablet_bootstrap.cc:654] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.969733 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20113491b108476b95b8f32b69176d0e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:58.971460 8833 raft_consensus.cc:482] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 98dd96f425fc4ab3a6a54f2dfaaf9fab)
I20260808 13:35:58.972112 8833 raft_consensus.cc:504] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:35:58.972595 5771 leader_election.cc:341] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:58.924743 6033 raft_consensus.cc:348] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.973505 6033 raft_consensus.cc:374] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:58.973860 6033 raft_consensus.cc:729] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:58.974669 6033 consensus_queue.cc:260] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:58.976980 6033 ts_tablet_manager.cc:1428] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.067s user 0.002s sys 0.003s
I20260808 13:35:58.982008 6026 tablet_bootstrap.cc:654] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:58.987082 8833 leader_election.cc:288] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:59.016201 6026 tablet_bootstrap.cc:492] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:59.016690 6026 ts_tablet_manager.cc:1397] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.060s user 0.000s sys 0.013s
I20260808 13:35:59.017175 5383 raft_consensus.cc:3037] T 1b9e8d48c55349058cf80bf7c2e149e7 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:58.996011 9052 leader_election.cc:288] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:35:59.030956 5283 leader_election.cc:341] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: 86c2b34c95244472beec43643e60ff5f
I20260808 13:35:59.032176 6026 raft_consensus.cc:348] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:59.033146 6026 raft_consensus.cc:374] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.033463 6026 raft_consensus.cc:729] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.034216 6026 consensus_queue.cc:260] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
W20260808 13:35:59.041467 9091 log.cc:924] Time spent T 62d68ab73bae4dd9af46e907b1fd198c P 946c9d8462644deba5b45fa724f1923c: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260808 13:35:59.052977 6033 tablet_bootstrap.cc:492] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:58.968604 9101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:59.069026 6046 tablet_bootstrap.cc:492] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:59.069459 6046 ts_tablet_manager.cc:1397] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.206s user 0.014s sys 0.039s
I20260808 13:35:59.072738 6046 raft_consensus.cc:348] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.073432 6046 raft_consensus.cc:374] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.073678 6046 raft_consensus.cc:729] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.074424 6046 consensus_queue.cc:260] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.091116 6046 ts_tablet_manager.cc:1428] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.021s user 0.005s sys 0.002s
I20260808 13:35:59.092155 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:35:59.092664 9101 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:35:59.093484 6046 tablet_bootstrap.cc:492] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:59.058354 6033 tablet_bootstrap.cc:654] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:59.096249 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:59.099062 6046 tablet_bootstrap.cc:654] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:59.103252 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:35:59.109818 9101 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:35:59.115576 9052 consensus_queue.cc:1035] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:35:59.136876 6033 tablet_bootstrap.cc:492] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:59.137382 6033 ts_tablet_manager.cc:1397] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.085s user 0.008s sys 0.003s
I20260808 13:35:59.152035 8567 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: Loaded TSK: 0
I20260808 13:35:59.153463 8567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:35:59.166988 6046 tablet_bootstrap.cc:492] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:59.167443 6046 ts_tablet_manager.cc:1397] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.076s user 0.022s sys 0.020s
I20260808 13:35:59.181612 6046 raft_consensus.cc:348] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.182333 6046 raft_consensus.cc:374] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.182582 6046 raft_consensus.cc:729] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.188911 6046 consensus_queue.cc:260] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.192152 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab26fa1872e4acf874f6577bded5e43" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.202986 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab26fa1872e4acf874f6577bded5e43" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.208021 6026 ts_tablet_manager.cc:1428] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.191s user 0.009s sys 0.004s
I20260808 13:35:59.225661 6026 tablet_bootstrap.cc:492] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:59.233165 6033 raft_consensus.cc:348] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.233997 6033 raft_consensus.cc:374] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.234262 6033 raft_consensus.cc:729] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.234926 6033 consensus_queue.cc:260] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.236550 6033 ts_tablet_manager.cc:1428] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.099s user 0.001s sys 0.012s
I20260808 13:35:59.237408 6033 tablet_bootstrap.cc:492] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:59.239341 6046 ts_tablet_manager.cc:1428] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.072s user 0.003s sys 0.004s
I20260808 13:35:59.254540 9092 raft_consensus.cc:482] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.255862 9092 raft_consensus.cc:504] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.261468 9092 leader_election.cc:288] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:59.279528 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70a5cf88d94b36913a6d1ccfcb979a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.281556 9116 raft_consensus.cc:482] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.282218 9116 raft_consensus.cc:504] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:59.283381 6046 tablet_bootstrap.cc:492] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:59.284904 9116 leader_election.cc:288] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:59.288722 6046 tablet_bootstrap.cc:654] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:59.288604 5550 leader_election.cc:341] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:59.299705 5550 leader_election.cc:341] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.312772 6026 tablet_bootstrap.cc:654] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:59.324415 6046 tablet_bootstrap.cc:492] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:59.324843 6046 ts_tablet_manager.cc:1397] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.003s
I20260808 13:35:59.328133 9116 raft_consensus.cc:482] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.328617 9116 raft_consensus.cc:504] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.334888 9117 raft_consensus.cc:482] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.335472 9117 raft_consensus.cc:504] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:59.337733 9117 leader_election.cc:288] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:59.337936 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20113491b108476b95b8f32b69176d0e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:59.338757 5644 raft_consensus.cc:3037] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.343024 9116 leader_election.cc:288] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:59.343171 6033 tablet_bootstrap.cc:654] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:59.344867 5285 leader_election.cc:341] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Not found: Tablet not found: 86c2b34c95244472beec43643e60ff5f
I20260808 13:35:59.345258 5285 leader_election.cc:302] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:59.346279 9043 raft_consensus.cc:2726] T 86c2b34c95244472beec43643e60ff5f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260808 13:35:59.348841 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 48525.3us
I20260808 13:35:59.374881 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d439b27c3ad34619972c71d652f1ff10" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.377401 6046 raft_consensus.cc:348] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.379518 6033 tablet_bootstrap.cc:492] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:59.379952 6033 ts_tablet_manager.cc:1397] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.143s user 0.008s sys 0.002s
I20260808 13:35:59.382345 6033 raft_consensus.cc:348] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.383378 9123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:35:59.384044 9123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:59.386193 9123 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:35:59.387737 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:35:59.392108 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:35:59.394995 4820 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:35:59.397558 5382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70a5cf88d94b36913a6d1ccfcb979a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.397854 9123 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:35:59.398739 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d439b27c3ad34619972c71d652f1ff10" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.399798 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3424ca753bcc4f448d314c56aba3323e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.400105 8929 consensus_queue.cc:1035] T 62d68ab73bae4dd9af46e907b1fd198c P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:59.400830 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1607745896141f68535b751dc211796" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.400159 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1607745896141f68535b751dc211796" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.404060 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3424ca753bcc4f448d314c56aba3323e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.404497 5644 raft_consensus.cc:2454] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:59.405030 9029 raft_consensus.cc:482] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.378552 6046 raft_consensus.cc:374] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.405376 6046 raft_consensus.cc:729] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.405462 9029 raft_consensus.cc:504] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.406068 6046 consensus_queue.cc:260] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.407414 9029 leader_election.cc:288] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:59.407653 6046 ts_tablet_manager.cc:1428] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.083s user 0.007s sys 0.000s
I20260808 13:35:59.409775 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007c7222c7bc48329420c740b31ddfae" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.410333 5386 raft_consensus.cc:2454] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:59.411738 5771 leader_election.cc:302] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:59.412495 9029 raft_consensus.cc:2781] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:59.412847 9029 raft_consensus.cc:482] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:59.413153 9029 raft_consensus.cc:3037] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.383033 6033 raft_consensus.cc:374] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.413815 6033 raft_consensus.cc:729] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.414752 6033 consensus_queue.cc:260] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.428812 6026 tablet_bootstrap.cc:492] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:59.429255 6026 ts_tablet_manager.cc:1397] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.204s user 0.011s sys 0.001s
I20260808 13:35:59.431865 6026 raft_consensus.cc:348] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.432520 6026 raft_consensus.cc:374] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.432816 6026 raft_consensus.cc:729] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.433372 6026 consensus_queue.cc:260] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.440984 9118 raft_consensus.cc:482] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.441431 9118 raft_consensus.cc:504] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.443377 9118 leader_election.cc:288] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:59.449615 5550 leader_election.cc:341] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.450855 9029 raft_consensus.cc:504] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:59.452862 4820 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 9dee4b3b14264f3aad0896f629ab5b27 (table default.Table_4cb4f6347fda4d268fd87065c9b063ea [id=11f8a63441444ac1a7aa7165cbe4b696], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9dee4b3b14264f3aad0896f629ab5b27 consensus state has no leader
W20260808 13:35:59.465133 5550 leader_election.cc:341] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:59.465708 5550 leader_election.cc:341] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:59.453886 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 54264.5us
I20260808 13:35:59.468725 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007c7222c7bc48329420c740b31ddfae" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:35:59.468677 9052 consensus_queue.cc:1035] T 356e15bbcd26409a9d6302599816e15f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:59.469221 5386 raft_consensus.cc:3037] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.470021 9029 leader_election.cc:288] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:59.471380 9116 raft_consensus.cc:482] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.471813 9116 raft_consensus.cc:504] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.473464 9043 raft_consensus.cc:482] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.473896 9043 raft_consensus.cc:504] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.475844 9043 leader_election.cc:288] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:59.463899 6033 ts_tablet_manager.cc:1428] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.084s user 0.005s sys 0.000s
I20260808 13:35:59.476204 5386 raft_consensus.cc:2454] T 007c7222c7bc48329420c740b31ddfae P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:35:59.480239 5771 leader_election.cc:302] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:59.482327 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3272708a835c432c878cc330e44490c5" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.483464 9129 raft_consensus.cc:2781] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:59.484156 9129 raft_consensus.cc:686] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:59.484967 9129 consensus_queue.cc:237] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.492271 6046 tablet_bootstrap.cc:492] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:59.493500 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3272708a835c432c878cc330e44490c5" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:59.494295 5637 raft_consensus.cc:2379] T 3272708a835c432c878cc330e44490c5 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:59.490095 9116 leader_election.cc:288] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:59.496681 5285 leader_election.cc:302] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:59.497697 9043 raft_consensus.cc:3037] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.498390 6046 tablet_bootstrap.cc:654] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:59.506641 6026 ts_tablet_manager.cc:1428] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.077s user 0.006s sys 0.000s
I20260808 13:35:59.516396 8744 consensus_queue.cc:1035] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:59.527953 9120 raft_consensus.cc:482] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.528465 9120 raft_consensus.cc:504] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.530695 6026 tablet_bootstrap.cc:492] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:59.530390 9120 leader_election.cc:288] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:59.534647 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 68457.9us
I20260808 13:35:59.536357 6033 tablet_bootstrap.cc:492] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:59.539618 9120 raft_consensus.cc:482] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.541172 9120 raft_consensus.cc:504] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.541647 6033 tablet_bootstrap.cc:654] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:59.546340 9120 leader_election.cc:288] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:35:59.549537 5551 leader_election.cc:341] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.549954 5551 leader_election.cc:302] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:59.551374 9118 raft_consensus.cc:2726] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:59.555567 8744 raft_consensus.cc:482] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.555487 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61dfd47db0a74ed8b7228b559dd68459" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.556119 8744 raft_consensus.cc:504] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.559072 8744 leader_election.cc:288] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:59.560135 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d2aae6405349fc887ec8527464e46a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:59.562077 5285 consensus_peers.cc:487] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba39706cff65404485959865fff1772a. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:35:59.563324 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7e8060cab44121acf6dadd2c2873b3" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:59.564005 5637 raft_consensus.cc:2454] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:59.564873 5285 leader_election.cc:302] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:59.565755 8744 raft_consensus.cc:2781] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:59.566120 8744 raft_consensus.cc:482] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:59.566506 8744 raft_consensus.cc:3037] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.566671 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7e8060cab44121acf6dadd2c2873b3" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.567348 5874 raft_consensus.cc:2454] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:59.578346 6033 tablet_bootstrap.cc:492] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:59.578795 6033 ts_tablet_manager.cc:1397] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.007s
I20260808 13:35:59.591120 9043 raft_consensus.cc:2726] T 3272708a835c432c878cc330e44490c5 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:59.594307 6046 tablet_bootstrap.cc:492] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:59.594794 6046 ts_tablet_manager.cc:1397] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.103s user 0.011s sys 0.000s
W20260808 13:35:59.598088 5550 leader_election.cc:341] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.598173 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2be108f26bb48d49216c6b7eef6b9b8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.602815 6026 tablet_bootstrap.cc:654] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:59.602651 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 51562.5us
W20260808 13:35:59.603670 5551 leader_election.cc:341] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.604022 5551 leader_election.cc:302] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:59.597021 6046 raft_consensus.cc:348] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.611140 6046 raft_consensus.cc:374] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.611466 6046 raft_consensus.cc:729] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.612231 6046 consensus_queue.cc:260] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.614701 6046 ts_tablet_manager.cc:1428] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260808 13:35:59.616397 4965 master_service.cc:946] ReplaceTablet: received request to replace tablet 433fcbc6cdca458b8404ae5862f280a3 from {username='slave'} at 127.0.0.1:49444
W20260808 13:35:59.616547 5550 leader_election.cc:341] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:35:59.617158 5550 leader_election.cc:341] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.619995 9118 raft_consensus.cc:2726] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:35:59.621301 5551 leader_election.cc:341] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.621678 5551 leader_election.cc:302] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:59.593853 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f53e5b5c4bd4a9385ad198b283c6e79" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:59.627796 5550 leader_election.cc:341] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.631150 6033 raft_consensus.cc:348] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.631778 6033 raft_consensus.cc:374] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.632020 6033 raft_consensus.cc:729] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.632157 9116 raft_consensus.cc:2726] T d439b27c3ad34619972c71d652f1ff10 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:59.632581 6033 consensus_queue.cc:260] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:59.633742 5551 leader_election.cc:341] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.634176 5551 leader_election.cc:302] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:59.636245 9118 raft_consensus.cc:2726] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:59.637988 6033 ts_tablet_manager.cc:1428] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.058s user 0.010s sys 0.000s
W20260808 13:35:59.639528 5551 leader_election.cc:341] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.642359 6033 tablet_bootstrap.cc:492] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:59.635368 8744 raft_consensus.cc:504] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.641111 6026 tablet_bootstrap.cc:492] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:35:59.643250 6026 ts_tablet_manager.cc:1397] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.113s user 0.010s sys 0.002s
I20260808 13:35:59.646045 6026 raft_consensus.cc:348] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.650677 6026 raft_consensus.cc:374] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.653450 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7e8060cab44121acf6dadd2c2873b3" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:59.653908 6046 tablet_bootstrap.cc:492] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:35:59.654027 6033 tablet_bootstrap.cc:654] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:59.654966 5875 raft_consensus.cc:3037] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.661470 5875 raft_consensus.cc:2454] T 7c7e8060cab44121acf6dadd2c2873b3 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:59.666728 6026 raft_consensus.cc:729] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.654507 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7e8060cab44121acf6dadd2c2873b3" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:59.667353 5637 raft_consensus.cc:3037] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.671797 9043 consensus_queue.cc:1035] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:59.674366 5637 raft_consensus.cc:2454] T 7c7e8060cab44121acf6dadd2c2873b3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:35:59.677047 5285 leader_election.cc:302] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:35:59.651017 5551 leader_election.cc:302] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:59.678233 9118 raft_consensus.cc:2726] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:59.679211 9138 raft_consensus.cc:2781] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:59.679642 9138 raft_consensus.cc:686] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:35:59.680423 9138 consensus_queue.cc:237] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:59.685511 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 46101.6us
I20260808 13:35:59.687731 8744 leader_election.cc:288] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:59.670653 9129 raft_consensus.cc:482] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.676395 6033 tablet_bootstrap.cc:492] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:59.689141 6033 ts_tablet_manager.cc:1397] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.047s user 0.006s sys 0.004s
I20260808 13:35:59.689003 9129 raft_consensus.cc:504] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.691031 9129 leader_election.cc:288] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:59.691481 6033 raft_consensus.cc:348] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.692144 6033 raft_consensus.cc:374] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.692377 6033 raft_consensus.cc:729] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.692919 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d12106f0d794a79935999a2b8dbe7e9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.686576 6026 consensus_queue.cc:260] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.692970 6033 consensus_queue.cc:260] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:35:59.694424 5771 leader_election.cc:341] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.695524 5772 leader_election.cc:302] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:59.696200 8929 raft_consensus.cc:2781] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:35:59.696604 8929 raft_consensus.cc:686] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:35:59.697309 8929 consensus_queue.cc:237] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.697599 8833 consensus_queue.cc:1035] T 9ea8f5322a404837b91e1d96749926b7 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:35:59.701241 6046 tablet_bootstrap.cc:654] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:59.703790 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61dfd47db0a74ed8b7228b559dd68459" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.704114 6033 ts_tablet_manager.cc:1428] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260808 13:35:59.711460 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d2aae6405349fc887ec8527464e46a" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.716296 6026 ts_tablet_manager.cc:1428] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.073s user 0.006s sys 0.000s
I20260808 13:35:59.717239 6026 tablet_bootstrap.cc:492] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:35:59.718580 6033 tablet_bootstrap.cc:492] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:59.721823 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2be108f26bb48d49216c6b7eef6b9b8" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.724227 6033 tablet_bootstrap.cc:654] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:59.726199 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f53e5b5c4bd4a9385ad198b283c6e79" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.738534 8929 raft_consensus.cc:482] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.739094 8929 raft_consensus.cc:504] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.741662 8929 leader_election.cc:288] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:59.743484 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:35:59.744642 5771 leader_election.cc:341] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.747778 9118 raft_consensus.cc:482] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.748272 9118 raft_consensus.cc:504] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:35:59.750289 9118 leader_election.cc:288] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:59.778875 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3451d72548a140c68a6ccc2b566a09b1" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:35:59.780611 5550 leader_election.cc:341] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.787251 8929 raft_consensus.cc:482] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.787705 8929 raft_consensus.cc:504] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.789558 8929 leader_election.cc:288] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:59.793793 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01ca2e4d6a2a4e2bb2dd63af8c125e9f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:35:59.797997 5379 raft_consensus.cc:2454] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:59.798827 5771 leader_election.cc:302] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:59.799500 9052 raft_consensus.cc:2781] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:59.799813 9052 raft_consensus.cc:482] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:59.800113 9052 raft_consensus.cc:3037] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.808674 9052 raft_consensus.cc:504] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.833354 5463 heartbeater.cc:503] Master 127.4.170.61:46555 was elected leader, sending a full tablet report...
I20260808 13:35:59.833972 9052 leader_election.cc:288] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:35:59.836109 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007c7222c7bc48329420c740b31ddfae" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:59.840253 5637 raft_consensus.cc:2454] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:59.857718 6033 tablet_bootstrap.cc:492] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:59.858182 6033 ts_tablet_manager.cc:1397] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.140s user 0.017s sys 0.016s
I20260808 13:35:59.860592 6033 raft_consensus.cc:348] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.861809 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007c7222c7bc48329420c740b31ddfae" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:59.863922 5637 raft_consensus.cc:3037] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.875429 6033 raft_consensus.cc:374] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.876900 6033 raft_consensus.cc:729] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.880533 6033 consensus_queue.cc:260] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.889772 6033 ts_tablet_manager.cc:1428] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.031s user 0.006s sys 0.001s
I20260808 13:35:59.890589 6033 tablet_bootstrap.cc:492] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:59.907920 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d12106f0d794a79935999a2b8dbe7e9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:59.908736 5635 raft_consensus.cc:2454] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:59.917711 6033 tablet_bootstrap.cc:654] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:35:59.919576 9161 log.cc:924] Time spent T 62d68ab73bae4dd9af46e907b1fd198c P 5cd987c3dbdb4168beb73cd7e577c7cb: Append to log took a long time: real 0.059s user 0.000s sys 0.000s
I20260808 13:35:59.922286 6046 tablet_bootstrap.cc:492] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:35:59.922695 6046 ts_tablet_manager.cc:1397] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.269s user 0.007s sys 0.004s
I20260808 13:35:59.922734 8744 raft_consensus.cc:482] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.923163 8744 raft_consensus.cc:504] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.857102 6026 tablet_bootstrap.cc:654] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:35:59.929685 6046 raft_consensus.cc:348] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.930565 6046 raft_consensus.cc:374] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.930852 6046 raft_consensus.cc:729] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.931526 6046 consensus_queue.cc:260] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.933210 6046 ts_tablet_manager.cc:1428] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260808 13:35:59.941469 6033 tablet_bootstrap.cc:492] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:35:59.941721 9167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:35:59.941898 6033 ts_tablet_manager.cc:1397] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.007s
I20260808 13:35:59.942567 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20113491b108476b95b8f32b69176d0e" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.944747 6033 raft_consensus.cc:348] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.945399 6033 raft_consensus.cc:374] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:35:59.945637 6033 raft_consensus.cc:729] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:35:59.946185 6033 consensus_queue.cc:260] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.948159 6033 ts_tablet_manager.cc:1428] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260808 13:35:59.948948 6033 tablet_bootstrap.cc:492] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:35:59.942564 9167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:35:59.927790 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:59.953853 5636 raft_consensus.cc:2454] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:59.953976 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20113491b108476b95b8f32b69176d0e" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:59.954756 5635 raft_consensus.cc:2379] T 20113491b108476b95b8f32b69176d0e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:35:59.957091 5285 leader_election.cc:302] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:35:59.957903 8833 raft_consensus.cc:3037] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.960348 9159 raft_consensus.cc:482] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.960754 9159 raft_consensus.cc:504] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.963914 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b314734698ed423ba9cf4134db5846ec" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.964555 5874 raft_consensus.cc:2454] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:59.965691 5283 leader_election.cc:302] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:35:59.966255 8833 raft_consensus.cc:2726] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:35:59.966816 9043 raft_consensus.cc:2781] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:59.967166 9043 raft_consensus.cc:482] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:59.967463 9043 raft_consensus.cc:3037] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.968148 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b314734698ed423ba9cf4134db5846ec" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:59.968880 9116 raft_consensus.cc:482] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:35:59.968827 5636 raft_consensus.cc:2454] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 0.
I20260808 13:35:59.969318 9116 raft_consensus.cc:504] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:35:59.972035 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01ca2e4d6a2a4e2bb2dd63af8c125e9f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:35:59.972638 5636 raft_consensus.cc:2454] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:35:59.974879 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01ca2e4d6a2a4e2bb2dd63af8c125e9f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:59.975378 5644 raft_consensus.cc:3037] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.980773 9043 raft_consensus.cc:504] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:35:59.984952 9159 leader_election.cc:288] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:35:59.987257 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b314734698ed423ba9cf4134db5846ec" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:35:59.987892 5635 raft_consensus.cc:3037] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.989115 9116 leader_election.cc:288] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:35:59.990872 5874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbdf03ebe4e48a98450e90a3a78181c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:35:59.991562 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b314734698ed423ba9cf4134db5846ec" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:35:59.992136 5869 raft_consensus.cc:3037] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:35:59.992988 5637 raft_consensus.cc:2454] T 007c7222c7bc48329420c740b31ddfae P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
W20260808 13:35:59.994191 5550 leader_election.cc:341] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:35:59.997057 5772 leader_election.cc:302] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:35:59.997913 8929 raft_consensus.cc:2781] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:35:59.998242 8929 raft_consensus.cc:482] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:35:59.998562 8929 raft_consensus.cc:3037] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:00.001509 6033 tablet_bootstrap.cc:654] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:00.005228 5644 raft_consensus.cc:2454] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
W20260808 13:36:00.012275 5551 leader_election.cc:341] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.017814 5551 leader_election.cc:302] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:00.017198 5772 leader_election.cc:302] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:36:00.010385 5869 raft_consensus.cc:2454] T b314734698ed423ba9cf4134db5846ec P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:36:00.018891 9029 raft_consensus.cc:2781] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:00.019783 5283 leader_election.cc:302] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
W20260808 13:36:00.020381 5551 leader_election.cc:341] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.020469 5635 raft_consensus.cc:2454] T b314734698ed423ba9cf4134db5846ec P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:36:00.019341 9029 raft_consensus.cc:482] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:00.020789 5551 leader_election.cc:302] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:00.020942 9029 raft_consensus.cc:3037] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:00.021476 9118 raft_consensus.cc:2726] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:00.022024 9118 raft_consensus.cc:2726] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:00.029757 5551 leader_election.cc:341] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.030149 5551 leader_election.cc:302] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:00.031502 6026 tablet_bootstrap.cc:492] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:00.031963 6026 ts_tablet_manager.cc:1397] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.315s user 0.012s sys 0.005s
I20260808 13:36:00.034861 8833 raft_consensus.cc:2781] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:00.035534 8833 raft_consensus.cc:686] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
I20260808 13:36:00.035475 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:36:00.036023 7741 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e672d7386144b299688abea9c64854c in term 4.
I20260808 13:36:00.036381 8833 consensus_queue.cc:237] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.038396 9118 raft_consensus.cc:2726] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:00.038354 4823 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e672d7386144b299688abea9c64854c, 87e8b888041b4135af55792050defcb5; no voters:
I20260808 13:36:00.038965 9043 leader_election.cc:288] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 election: Requested vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:00.039443 5372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3451d72548a140c68a6ccc2b566a09b1" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:00.045593 6046 tablet_bootstrap.cc:492] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
W20260808 13:36:00.048118 5551 leader_election.cc:341] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.048655 9118 raft_consensus.cc:482] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:00.049310 9118 raft_consensus.cc:504] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.049084 5551 leader_election.cc:302] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:00.051434 9118 leader_election.cc:288] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:00.052872 9118 raft_consensus.cc:2726] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:00.055810 5551 leader_election.cc:341] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.056208 5551 leader_election.cc:302] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:00.056603 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f0a6d9bd2c74d0790badd7977f027e3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:00.058279 5550 leader_election.cc:341] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.067078 9118 raft_consensus.cc:2726] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:00.068025 6026 raft_consensus.cc:348] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.068707 6026 raft_consensus.cc:374] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:00.068944 6026 raft_consensus.cc:729] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:00.028647 4961 catalog_manager.cc:5495] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3). New cstate: current_term: 1 leader_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } }
I20260808 13:36:00.070502 4961 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet 37495b29698940edb2924eb612e94bd7 (table default.Table_45fdb943ae1b499e98db51a5ee6589a2 [id=ed0521e8ecae4c609b2a7426e5a565c0]). Expected version 1 got 0
I20260808 13:36:00.072314 9167 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:36:00.074381 9167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260808 13:36:00.074834 9052 consensus_queue.cc:1035] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:00.078199 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:36:00.084230 9167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:36:00.084710 9167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 4 FOLLOWER]: Advancing to term 5
I20260808 13:36:00.089313 5772 leader_election.cc:302] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:00.090021 6026 consensus_queue.cc:260] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.092561 6026 ts_tablet_manager.cc:1428] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.060s user 0.005s sys 0.001s
I20260808 13:36:00.093462 4961 catalog_manager.cc:5587] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577) does not have the latest schema for tablet ffbdf03ebe4e48a98450e90a3a78181c (table default.Table_510c442cafba44a9bf48a292ca01fbd4 [id=212e1314b3c04c4d8dd06d5730fa25cf]). Expected version 1 got 0
I20260808 13:36:00.093778 9177 raft_consensus.cc:2781] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:00.094203 9177 raft_consensus.cc:686] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:00.094969 9177 consensus_queue.cc:237] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.083576 9169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:36:00.105476 9169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:36:00.106043 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbdf03ebe4e48a98450e90a3a78181c" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:00.117484 8929 raft_consensus.cc:504] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.128471 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:36:00.128703 9169 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:36:00.128816 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:36:00.131260 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:36:00.132426 6046 tablet_bootstrap.cc:654] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:00.136125 8929 leader_election.cc:288] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:00.123906 9029 raft_consensus.cc:504] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.139369 6026 tablet_bootstrap.cc:492] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:00.140977 9029 leader_election.cc:288] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:00.144675 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d12106f0d794a79935999a2b8dbe7e9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:00.145263 5638 raft_consensus.cc:3037] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:36:00.149034 5551 leader_election.cc:341] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.149425 5551 leader_election.cc:302] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:00.151599 5371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f0a6d9bd2c74d0790badd7977f027e3" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:00.152711 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d12106f0d794a79935999a2b8dbe7e9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:36:00.153245 4960 catalog_manager.cc:5587] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173) does not have the latest schema for tablet 42218e4330d147f389717108bfe0acea (table default.Table_660efdc735cf4c31bf966be22b38dc28 [id=f603e442844e4a09bfecb79b93014265]). Expected version 1 got 0
I20260808 13:36:00.155751 9169 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260808 13:36:00.157171 5771 leader_election.cc:341] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.161659 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:36:00.164124 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01ca2e4d6a2a4e2bb2dd63af8c125e9f" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:36:00.164705 5379 raft_consensus.cc:3037] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:00.170890 9118 raft_consensus.cc:2726] T ffbdf03ebe4e48a98450e90a3a78181c P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:00.176285 6026 tablet_bootstrap.cc:654] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:36:00.184053 5551 leader_election.cc:341] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.184466 5551 leader_election.cc:302] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:36:00.185016 5283 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 145927us
I20260808 13:36:00.194736 9118 raft_consensus.cc:2726] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:00.199227 5771 leader_election.cc:341] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.182037 9167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:36:00.204653 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7"
I20260808 13:36:00.207031 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 5 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5"
I20260808 13:36:00.210085 9167 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 election: Requested vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:36:00.211136 4820 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:36:00.212410 9167 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260808 13:36:00.214990 6026 tablet_bootstrap.cc:492] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:00.215427 6026 ts_tablet_manager.cc:1397] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.076s user 0.008s sys 0.002s
I20260808 13:36:00.047762 8744 leader_election.cc:288] T 20113491b108476b95b8f32b69176d0e P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:00.217702 6026 raft_consensus.cc:348] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.218343 6026 raft_consensus.cc:374] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:00.218580 6026 raft_consensus.cc:729] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:00.221092 9176 consensus_queue.cc:1035] T 3272708a835c432c878cc330e44490c5 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:00.222162 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:00.222718 5635 raft_consensus.cc:3037] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:00.223361 6033 tablet_bootstrap.cc:492] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:00.223835 6033 ts_tablet_manager.cc:1397] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.275s user 0.026s sys 0.011s
I20260808 13:36:00.229285 5635 raft_consensus.cc:2454] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:00.230432 6026 consensus_queue.cc:260] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:36:00.233948 4943 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 42218e4330d147f389717108bfe0acea (table default.Table_660efdc735cf4c31bf966be22b38dc28 [id=f603e442844e4a09bfecb79b93014265], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 42218e4330d147f389717108bfe0acea consensus state has no leader
I20260808 13:36:00.234265 5638 raft_consensus.cc:2454] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
W20260808 13:36:00.243889 4960 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 42218e4330d147f389717108bfe0acea (table default.Table_660efdc735cf4c31bf966be22b38dc28 [id=f603e442844e4a09bfecb79b93014265], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 42218e4330d147f389717108bfe0acea consensus state has no leader
I20260808 13:36:00.267521 5950 heartbeater.cc:503] Master 127.4.170.61:46555 was elected leader, sending a full tablet report...
I20260808 13:36:00.280463 5379 raft_consensus.cc:2454] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:00.283093 6046 tablet_bootstrap.cc:492] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:36:00.283654 6046 ts_tablet_manager.cc:1397] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.238s user 0.005s sys 0.011s
I20260808 13:36:00.285959 6046 raft_consensus.cc:348] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.260211 6026 ts_tablet_manager.cc:1428] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.044s user 0.004s sys 0.000s
I20260808 13:36:00.287626 6026 tablet_bootstrap.cc:492] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:00.304690 6026 tablet_bootstrap.cc:654] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:00.308493 9177 consensus_queue.cc:1035] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:00.226153 6033 raft_consensus.cc:348] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.315081 6033 raft_consensus.cc:374] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:00.323094 6033 raft_consensus.cc:729] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:00.324266 6033 consensus_queue.cc:260] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.290172 5335 tablet_service.cc:1514] Processing DeleteTablet for tablet ba39706cff65404485959865fff1772a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:38994
I20260808 13:36:00.341951 6046 raft_consensus.cc:374] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:00.342269 6046 raft_consensus.cc:729] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:36:00.326393 5772 leader_election.cc:302] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:36:00.351079 8929 raft_consensus.cc:2781] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:00.351583 8929 raft_consensus.cc:686] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:00.352442 8929 consensus_queue.cc:237] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.359207 5772 leader_election.cc:302] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:36:00.360985 9204 tablet_replica.cc:330] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c: stopping tablet replica
I20260808 13:36:00.366916 9204 raft_consensus.cc:2229] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:36:00.367985 9204 raft_consensus.cc:2258] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:00.375849 9029 raft_consensus.cc:2781] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:00.376353 9029 raft_consensus.cc:686] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:00.377111 9029 consensus_queue.cc:237] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.381891 9204 ts_tablet_manager.cc:1905] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:00.390969 6026 tablet_bootstrap.cc:492] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:00.391386 6026 ts_tablet_manager.cc:1397] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.104s user 0.006s sys 0.005s
I20260808 13:36:00.414801 6033 ts_tablet_manager.cc:1428] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.191s user 0.006s sys 0.000s
I20260808 13:36:00.415748 6033 tablet_bootstrap.cc:492] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:00.417217 6026 raft_consensus.cc:348] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.417869 6026 raft_consensus.cc:374] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:00.418115 6026 raft_consensus.cc:729] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:00.420370 9116 raft_consensus.cc:482] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:00.420822 9116 raft_consensus.cc:504] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.343057 6046 consensus_queue.cc:260] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.422947 9116 leader_election.cc:288] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:00.424114 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "553a01330cfe4dd38a61860dc943fbbc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:00.424563 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "553a01330cfe4dd38a61860dc943fbbc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:00.425264 5551 leader_election.cc:341] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:36:00.425980 5550 leader_election.cc:341] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.426326 5550 leader_election.cc:302] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:00.426959 9116 raft_consensus.cc:2726] T 553a01330cfe4dd38a61860dc943fbbc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:00.429167 9204 ts_tablet_manager.cc:1918] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:00.428870 6026 consensus_queue.cc:260] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.434144 9204 log.cc:1196] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/ba39706cff65404485959865fff1772a
I20260808 13:36:00.435433 9204 ts_tablet_manager.cc:1939] T ba39706cff65404485959865fff1772a P 946c9d8462644deba5b45fa724f1923c: Deleting consensus metadata
I20260808 13:36:00.444895 4942 catalog_manager.cc:4841] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): tablet ba39706cff65404485959865fff1772a (table default.Table_fc7cb305e35541c781c3e3790cf8a0af [id=b885c5ab382c45809d6f896ebf8771bd]) successfully deleted
I20260808 13:36:00.422780 6046 ts_tablet_manager.cc:1428] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.139s user 0.004s sys 0.001s
I20260808 13:36:00.446954 6046 tablet_bootstrap.cc:492] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:36:00.467515 4864 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 5 FOLLOWER]: Rejecting Update request from peer 63c875fc5bcd41a9b1bcf3bd4ae321d7 for earlier term 4. Current term is 5. Ops: [4.596-4.596]
I20260808 13:36:00.454365 6026 ts_tablet_manager.cc:1428] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.063s user 0.005s sys 0.000s
I20260808 13:36:00.481261 9173 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 }, Status: INVALID_TERM, Last received: 4.595, Next index: 596, Last known committed idx: 595, Time since last communication: 0.009s
I20260808 13:36:00.486009 6026 tablet_bootstrap.cc:492] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:00.501510 9168 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 4 LEADER]: Stepping down as leader of term 4
I20260808 13:36:00.502081 9168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 63c875fc5bcd41a9b1bcf3bd4ae321d7, State: Running, Role: LEADER
I20260808 13:36:00.504323 9168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 595, Committed index: 595, Last appended: 4.597, Last appended by leader: 597, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:36:00.506749 9168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 4 FOLLOWER]: Advancing to term 5
I20260808 13:36:00.528630 6026 tablet_bootstrap.cc:654] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:00.529629 8833 consensus_queue.cc:1035] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:00.565369 9176 raft_consensus.cc:482] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:00.565848 9176 raft_consensus.cc:504] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.567922 9176 leader_election.cc:288] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:00.570303 9176 consensus_queue.cc:1035] T 007c7222c7bc48329420c740b31ddfae P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:00.577534 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3424ca753bcc4f448d314c56aba3323e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:00.578130 5379 raft_consensus.cc:2454] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:00.579353 5771 leader_election.cc:302] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:00.584203 9176 raft_consensus.cc:2781] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:00.584708 9176 raft_consensus.cc:482] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:00.585255 9176 raft_consensus.cc:3037] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:00.620743 6026 tablet_bootstrap.cc:492] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:00.621167 6026 ts_tablet_manager.cc:1397] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.136s user 0.005s sys 0.006s
I20260808 13:36:00.624269 6026 raft_consensus.cc:348] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.625041 6026 raft_consensus.cc:374] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:00.625310 6026 raft_consensus.cc:729] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:00.652007 6026 consensus_queue.cc:260] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.654524 6026 ts_tablet_manager.cc:1428] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.033s user 0.006s sys 0.001s
I20260808 13:36:00.663736 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3424ca753bcc4f448d314c56aba3323e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:00.664374 5630 raft_consensus.cc:2454] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:00.667418 9188 consensus_queue.cc:1035] T 7c7e8060cab44121acf6dadd2c2873b3 P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:00.669633 6026 tablet_bootstrap.cc:492] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:00.670938 5831 tablet_service.cc:1514] Processing DeleteTablet for tablet ba39706cff65404485959865fff1772a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:29 UTC) from {username='slave'} at 127.0.0.1:48546
I20260808 13:36:00.671957 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d12106f0d794a79935999a2b8dbe7e9 with delete_type TABLET_DATA_DELETED (replaced by tablet 30691adffb1f40bdaeaa24f978a38bd7) from {username='slave'} at 127.0.0.1:48546
I20260808 13:36:00.680274 6033 tablet_bootstrap.cc:654] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:00.684369 6026 tablet_bootstrap.cc:654] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:00.688143 9221 tablet_replica.cc:330] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:36:00.699415 9221 raft_consensus.cc:2229] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:00.700029 9221 raft_consensus.cc:2258] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:00.702057 9221 ts_tablet_manager.cc:1905] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:00.707473 9206 raft_consensus.cc:482] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:00.707995 9206 raft_consensus.cc:504] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.711082 9206 leader_election.cc:288] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:00.711035 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9ea181156f64388ade8a07708b2e814" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:00.712572 5550 leader_election.cc:341] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.712807 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9ea181156f64388ade8a07708b2e814" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:36:00.714756 5551 leader_election.cc:341] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.715137 5551 leader_election.cc:302] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:00.716650 9206 raft_consensus.cc:2726] T a9ea181156f64388ade8a07708b2e814 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:00.674425 6046 tablet_bootstrap.cc:654] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:00.727185 6026 tablet_bootstrap.cc:492] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:00.727623 6026 ts_tablet_manager.cc:1397] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.058s user 0.008s sys 0.004s
I20260808 13:36:00.730130 6026 raft_consensus.cc:348] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.730825 6026 raft_consensus.cc:374] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:00.731076 6026 raft_consensus.cc:729] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:00.731704 6026 consensus_queue.cc:260] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.734157 6026 ts_tablet_manager.cc:1428] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 13:36:00.736572 9206 raft_consensus.cc:482] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:00.737036 9206 raft_consensus.cc:504] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.739089 9221 ts_tablet_manager.cc:1918] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:00.741279 9206 leader_election.cc:288] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:00.742246 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41347e0da1654dde8b48c896db6a0915" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:00.741199 6026 tablet_bootstrap.cc:492] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:00.742950 9221 log.cc:1196] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/ba39706cff65404485959865fff1772a
I20260808 13:36:00.743270 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41347e0da1654dde8b48c896db6a0915" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:00.743551 5551 leader_election.cc:341] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.744024 9221 ts_tablet_manager.cc:1939] T ba39706cff65404485959865fff1772a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
W20260808 13:36:00.744731 5550 leader_election.cc:341] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:00.745106 5550 leader_election.cc:302] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:00.748337 9206 raft_consensus.cc:2726] T 41347e0da1654dde8b48c896db6a0915 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:00.765631 6046 tablet_bootstrap.cc:492] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:36:00.766093 6046 ts_tablet_manager.cc:1397] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.319s user 0.010s sys 0.000s
I20260808 13:36:00.768493 6046 raft_consensus.cc:348] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.769227 6046 raft_consensus.cc:374] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:00.769505 6046 raft_consensus.cc:729] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:36:00.706589 9176 raft_consensus.cc:504] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.790411 9177 raft_consensus.cc:482] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:00.790936 9177 raft_consensus.cc:504] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.792764 9177 leader_election.cc:288] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:00.794651 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f778e7fa51414cb209fbe1aea92739" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:00.796415 9221 tablet_replica.cc:330] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: stopping tablet replica
I20260808 13:36:00.797348 9221 raft_consensus.cc:2229] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:36:00.797482 4941 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet ba39706cff65404485959865fff1772a (table default.Table_fc7cb305e35541c781c3e3790cf8a0af [id=b885c5ab382c45809d6f896ebf8771bd]) successfully deleted
I20260808 13:36:00.798041 9221 pending_rounds.cc:62] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Trying to abort 1 pending ops.
I20260808 13:36:00.798274 9221 pending_rounds.cc:68] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259472948277248 op_type: NO_OP noop_request { }
I20260808 13:36:00.798738 9221 raft_consensus.cc:2866] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:36:00.799070 9221 raft_consensus.cc:2258] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:00.801507 9221 ts_tablet_manager.cc:1905] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:00.810966 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f778e7fa51414cb209fbe1aea92739" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:00.811617 5637 raft_consensus.cc:2454] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:00.817183 9176 leader_election.cc:288] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:00.817955 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3424ca753bcc4f448d314c56aba3323e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:36:00.818574 5379 raft_consensus.cc:3037] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:00.820475 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3424ca753bcc4f448d314c56aba3323e" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:00.821030 5637 raft_consensus.cc:3037] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:00.825151 6033 tablet_bootstrap.cc:492] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:00.825259 5379 raft_consensus.cc:2454] T 3424ca753bcc4f448d314c56aba3323e P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:00.825922 6033 ts_tablet_manager.cc:1397] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.410s user 0.010s sys 0.003s
I20260808 13:36:00.826666 5771 leader_election.cc:302] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:00.827500 9129 raft_consensus.cc:2781] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:00.827989 9129 raft_consensus.cc:686] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:00.828298 6033 raft_consensus.cc:348] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.831051 6033 raft_consensus.cc:374] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:00.830940 6026 tablet_bootstrap.cc:654] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:00.831341 6033 raft_consensus.cc:729] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:00.828833 9129 consensus_queue.cc:237] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.831919 6033 consensus_queue.cc:260] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.833979 6033 ts_tablet_manager.cc:1428] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.008s user 0.001s sys 0.002s
I20260808 13:36:00.847872 5772 leader_election.cc:302] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:00.849033 9177 raft_consensus.cc:2781] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:00.849419 9177 raft_consensus.cc:482] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:00.849745 9177 raft_consensus.cc:3037] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:00.869400 6026 tablet_bootstrap.cc:492] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:00.869119 9177 raft_consensus.cc:504] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.869831 6026 ts_tablet_manager.cc:1397] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.129s user 0.009s sys 0.004s
I20260808 13:36:00.872371 6026 raft_consensus.cc:348] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.873034 6026 raft_consensus.cc:374] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:00.873281 6026 raft_consensus.cc:729] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:00.873981 6026 consensus_queue.cc:260] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:00.882565 5637 raft_consensus.cc:2454] T 3424ca753bcc4f448d314c56aba3323e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:00.895763 9177 leader_election.cc:288] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:00.897161 6026 ts_tablet_manager.cc:1428] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.027s user 0.003s sys 0.003s
I20260808 13:36:00.897104 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f778e7fa51414cb209fbe1aea92739" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:00.897675 5630 raft_consensus.cc:3037] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:00.898478 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f778e7fa51414cb209fbe1aea92739" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:36:00.907189 6046 consensus_queue.cc:260] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:00.908416 5376 raft_consensus.cc:2454] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:00.919643 9129 consensus_queue.cc:1035] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:00.924649 6026 tablet_bootstrap.cc:492] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:00.932097 6033 tablet_bootstrap.cc:492] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:00.910385 6046 ts_tablet_manager.cc:1428] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.144s user 0.004s sys 0.004s
I20260808 13:36:00.914319 5630 raft_consensus.cc:2454] T b0f778e7fa51414cb209fbe1aea92739 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:00.935407 6046 tablet_bootstrap.cc:492] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:36:00.939966 9175 consensus_queue.cc:1035] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:00.952440 6026 tablet_bootstrap.cc:654] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:00.953893 9177 consensus_queue.cc:1035] T 20113491b108476b95b8f32b69176d0e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:00.966570 9230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:00.967141 9230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:36:00.970201 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 6 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:36:00.971076 4977 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e672d7386144b299688abea9c64854c for term 6 because replica has last-logged OpId of term: 4 index: 597, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 595.
I20260808 13:36:00.972725 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 6 candidate_status { last_received { term: 4 index: 595 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:36:00.974013 7741 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e672d7386144b299688abea9c64854c for term 6 because replica has last-logged OpId of term: 4 index: 596, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 595.
I20260808 13:36:00.954950 6033 tablet_bootstrap.cc:654] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:00.955370 5772 leader_election.cc:302] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:36:00.978209 4820 leader_election.cc:302] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e672d7386144b299688abea9c64854c; no voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5
I20260808 13:36:01.000322 9240 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260808 13:36:00.988379 9129 raft_consensus.cc:2781] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:01.003688 9230 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:36:01.012408 6026 tablet_bootstrap.cc:492] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:01.013541 6026 ts_tablet_manager.cc:1397] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.089s user 0.007s sys 0.003s
I20260808 13:36:01.017119 6026 raft_consensus.cc:348] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.026769 6026 raft_consensus.cc:374] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.029191 6026 raft_consensus.cc:729] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.029953 6026 consensus_queue.cc:260] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.032892 6026 ts_tablet_manager.cc:1428] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260808 13:36:01.034639 8744 consensus_queue.cc:1035] T b314734698ed423ba9cf4134db5846ec P 946c9d8462644deba5b45fa724f1923c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:01.039474 6046 tablet_bootstrap.cc:654] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.042547 9237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:36:01.043401 9237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:36:01.043721 9221 ts_tablet_manager.cc:1918] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:01.046661 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 596 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:36:01.047417 4864 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 87e8b888041b4135af55792050defcb5 in current term 5: Already voted for candidate 3e672d7386144b299688abea9c64854c in this term.
I20260808 13:36:01.048003 6026 tablet_bootstrap.cc:492] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:01.053512 6026 tablet_bootstrap.cc:654] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.056458 9129 raft_consensus.cc:686] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:01.057307 9129 consensus_queue.cc:237] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.044348 9221 log.cc:1196] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/8d12106f0d794a79935999a2b8dbe7e9
I20260808 13:36:01.062433 9221 ts_tablet_manager.cc:1939] T 8d12106f0d794a79935999a2b8dbe7e9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Deleting consensus metadata
I20260808 13:36:01.071070 9206 raft_consensus.cc:482] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.071549 9206 raft_consensus.cc:504] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.075888 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7f6487e28f4aaa94d448be71f8a3c2" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:01.077062 5550 leader_election.cc:341] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.077920 4941 catalog_manager.cc:4841] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): tablet 8d12106f0d794a79935999a2b8dbe7e9 (table default.Table_c2023a99a0654190adb8ada69a32f740 [id=054085276211478a8982d48b0599a684]) successfully deleted
I20260808 13:36:01.076468 9206 leader_election.cc:288] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:01.082718 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7f6487e28f4aaa94d448be71f8a3c2" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:36:01.085206 5551 leader_election.cc:341] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.085563 5551 leader_election.cc:302] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:01.086727 9206 raft_consensus.cc:2726] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:01.093729 6026 tablet_bootstrap.cc:492] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:01.094197 6026 ts_tablet_manager.cc:1397] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.004s
I20260808 13:36:01.096576 6026 raft_consensus.cc:348] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.097286 6026 raft_consensus.cc:374] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.097546 6026 raft_consensus.cc:729] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.101902 6026 consensus_queue.cc:260] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.103917 6026 ts_tablet_manager.cc:1428] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260808 13:36:01.104796 6026 tablet_bootstrap.cc:492] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:01.126930 9176 raft_consensus.cc:482] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.127383 9176 raft_consensus.cc:504] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.130982 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f53e5b5c4bd4a9385ad198b283c6e79" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:01.131109 9176 leader_election.cc:288] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:36:01.132296 5771 leader_election.cc:341] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.133380 6026 tablet_bootstrap.cc:654] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.146658 9237 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:36:01.149998 6046 tablet_bootstrap.cc:492] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:36:01.150490 6046 ts_tablet_manager.cc:1397] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.215s user 0.008s sys 0.000s
I20260808 13:36:01.152894 6046 raft_consensus.cc:348] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.153631 6046 raft_consensus.cc:374] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.153908 6046 raft_consensus.cc:729] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.153961 9251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.154505 9251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:36:01.160194 9251 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:36:01.161373 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 6 candidate_status { last_received { term: 4 index: 597 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
I20260808 13:36:01.162042 4864 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63c875fc5bcd41a9b1bcf3bd4ae321d7 in term 5.
I20260808 13:36:01.163547 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 6 candidate_status { last_received { term: 4 index: 597 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:36:01.164161 7741 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63c875fc5bcd41a9b1bcf3bd4ae321d7 in term 4.
I20260808 13:36:01.168455 6026 tablet_bootstrap.cc:492] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:01.168992 6026 ts_tablet_manager.cc:1397] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.064s user 0.008s sys 0.000s
I20260808 13:36:01.170704 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 5 candidate_status { last_received { term: 4 index: 596 } } ignore_live_leader: false dest_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" is_pre_election: true
I20260808 13:36:01.171651 4977 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 87e8b888041b4135af55792050defcb5 for term 5 because replica has last-logged OpId of term: 4 index: 597, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 596.
I20260808 13:36:01.172883 9129 consensus_queue.cc:1035] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:01.173698 6026 raft_consensus.cc:348] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.174544 6026 raft_consensus.cc:374] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.174853 6026 raft_consensus.cc:729] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.179208 6046 consensus_queue.cc:260] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.179864 4944 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63c875fc5bcd41a9b1bcf3bd4ae321d7, 87e8b888041b4135af55792050defcb5; no voters:
I20260808 13:36:01.182955 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87e8b888041b4135af55792050defcb5; no voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7
I20260808 13:36:01.183002 6026 consensus_queue.cc:260] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.185514 6046 ts_tablet_manager.cc:1428] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.035s user 0.004s sys 0.001s
I20260808 13:36:01.187844 6026 ts_tablet_manager.cc:1428] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.019s user 0.007s sys 0.001s
I20260808 13:36:01.190064 6033 tablet_bootstrap.cc:492] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:01.190560 6033 ts_tablet_manager.cc:1397] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.259s user 0.004s sys 0.004s
I20260808 13:36:01.192847 6033 raft_consensus.cc:348] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.193518 6033 raft_consensus.cc:374] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.193759 6033 raft_consensus.cc:729] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.201587 9237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 4 FOLLOWER]: Advancing to term 5
I20260808 13:36:01.205268 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f53e5b5c4bd4a9385ad198b283c6e79" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:01.206004 5630 raft_consensus.cc:2454] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:01.194346 6033 consensus_queue.cc:260] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.218418 6033 ts_tablet_manager.cc:1428] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260808 13:36:01.231143 9029 consensus_queue.cc:1035] T 01ca2e4d6a2a4e2bb2dd63af8c125e9f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:01.231400 5772 leader_election.cc:302] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:36:01.232884 9029 raft_consensus.cc:2781] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:01.233215 9029 raft_consensus.cc:482] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:01.233513 9029 raft_consensus.cc:3037] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:01.239648 6026 tablet_bootstrap.cc:492] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:01.243013 9206 raft_consensus.cc:482] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.243510 9206 raft_consensus.cc:504] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.246245 9206 leader_election.cc:288] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:01.249312 6026 tablet_bootstrap.cc:654] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:36:01.251310 4819 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 42218e4330d147f389717108bfe0acea (table default.Table_660efdc735cf4c31bf966be22b38dc28 [id=f603e442844e4a09bfecb79b93014265], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 42218e4330d147f389717108bfe0acea consensus state has no leader
I20260808 13:36:01.252549 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a11a2666f0b44d70a08a9195e8fcf69d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:01.253608 9206 raft_consensus.cc:482] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.254109 9206 raft_consensus.cc:504] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:36:01.253943 5551 leader_election.cc:341] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.256873 9206 leader_election.cc:288] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:01.257571 6046 tablet_bootstrap.cc:492] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:36:01.262857 6046 tablet_bootstrap.cc:654] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.264989 9029 raft_consensus.cc:504] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.267658 9251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260808 13:36:01.268356 9251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:01.268847 9251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 5 FOLLOWER]: Advancing to term 6
I20260808 13:36:01.281816 6026 tablet_bootstrap.cc:492] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:01.282234 6026 ts_tablet_manager.cc:1397] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.043s user 0.005s sys 0.004s
I20260808 13:36:01.282418 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c50fd06b8841c8b5a944401b495d2e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:36:01.284196 5551 leader_election.cc:341] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.284844 6026 raft_consensus.cc:348] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.285566 6026 raft_consensus.cc:374] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.285835 6026 raft_consensus.cc:729] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.286515 6026 consensus_queue.cc:260] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.288595 6026 ts_tablet_manager.cc:1428] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 13:36:01.289615 6026 tablet_bootstrap.cc:492] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:01.305640 5376 raft_consensus.cc:3037] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:01.309759 6046 tablet_bootstrap.cc:492] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:36:01.310205 6046 ts_tablet_manager.cc:1397] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.053s user 0.010s sys 0.008s
I20260808 13:36:01.314412 6046 raft_consensus.cc:348] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.315323 6046 raft_consensus.cc:374] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.315573 6046 raft_consensus.cc:729] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
W20260808 13:36:01.316366 4943 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 42218e4330d147f389717108bfe0acea (table default.Table_660efdc735cf4c31bf966be22b38dc28 [id=f603e442844e4a09bfecb79b93014265], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 42218e4330d147f389717108bfe0acea consensus state has no leader
I20260808 13:36:01.316124 6046 consensus_queue.cc:260] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.338397 9029 leader_election.cc:288] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:01.341140 6046 ts_tablet_manager.cc:1428] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260808 13:36:01.341506 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f53e5b5c4bd4a9385ad198b283c6e79" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:36:01.342911 9206 raft_consensus.cc:482] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.343375 9206 raft_consensus.cc:504] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.352557 6046 tablet_bootstrap.cc:492] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:36:01.357945 9237 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:36:01.351547 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a11a2666f0b44d70a08a9195e8fcf69d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:01.354379 5771 leader_election.cc:341] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.356690 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c50fd06b8841c8b5a944401b495d2e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:36:01.370195 6046 tablet_bootstrap.cc:654] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.371636 6033 tablet_bootstrap.cc:492] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:01.372040 9251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:36:01.374884 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 6 candidate_status { last_received { term: 4 index: 597 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c"
I20260808 13:36:01.375507 4864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 5 FOLLOWER]: Advancing to term 6
I20260808 13:36:01.384316 4864 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63c875fc5bcd41a9b1bcf3bd4ae321d7 in term 6.
I20260808 13:36:01.386193 4941 leader_election.cc:302] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e672d7386144b299688abea9c64854c, 63c875fc5bcd41a9b1bcf3bd4ae321d7; no voters:
I20260808 13:36:01.390244 6033 tablet_bootstrap.cc:654] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.396662 9263 raft_consensus.cc:482] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.397171 9263 raft_consensus.cc:504] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.409457 9271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 6 FOLLOWER]: Leader election won for term 6
I20260808 13:36:01.412279 6046 tablet_bootstrap.cc:492] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:36:01.412667 6046 ts_tablet_manager.cc:1397] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.060s user 0.007s sys 0.004s
I20260808 13:36:01.414258 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f53e5b5c4bd4a9385ad198b283c6e79" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:01.414801 5644 raft_consensus.cc:3037] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:01.419479 6046 raft_consensus.cc:348] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.420136 6046 raft_consensus.cc:374] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.420395 6046 raft_consensus.cc:729] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.423563 9175 raft_consensus.cc:482] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.424054 9175 raft_consensus.cc:504] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.429327 9175 leader_election.cc:288] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:01.429523 6046 consensus_queue.cc:260] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.436560 6046 ts_tablet_manager.cc:1428] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.023s user 0.006s sys 0.001s
I20260808 13:36:01.437217 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:36:01.469698 6033 tablet_bootstrap.cc:492] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:01.470116 6033 ts_tablet_manager.cc:1397] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.099s user 0.015s sys 0.006s
I20260808 13:36:01.471503 9245 consensus_queue.cc:1035] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:36:01.479427 6033 raft_consensus.cc:348] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.480111 6033 raft_consensus.cc:374] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.480337 6033 raft_consensus.cc:729] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.482511 6033 consensus_queue.cc:260] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.484687 9271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [term 6 LEADER]: Becoming Leader. State: Replica: 63c875fc5bcd41a9b1bcf3bd4ae321d7, State: Running, Role: LEADER
I20260808 13:36:01.485981 9271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 595, Committed index: 595, Last appended: 4.597, Last appended by leader: 597, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:36:01.495132 6033 ts_tablet_manager.cc:1428] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.025s user 0.005s sys 0.003s
I20260808 13:36:01.497287 9273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c875fc5bcd41a9b1bcf3bd4ae321d7. Latest consensus state: current_term: 6 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:36:01.509925 6033 tablet_bootstrap.cc:492] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:01.523221 9206 leader_election.cc:288] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:01.527791 9263 leader_election.cc:288] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:01.530920 6033 tablet_bootstrap.cc:654] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.558822 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:01.561872 5638 raft_consensus.cc:2454] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 1.
I20260808 13:36:01.563742 5644 raft_consensus.cc:2454] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:01.568113 6033 tablet_bootstrap.cc:492] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:01.568568 6033 ts_tablet_manager.cc:1397] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.059s user 0.008s sys 0.006s
I20260808 13:36:01.571527 5285 leader_election.cc:302] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters: 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:01.572347 6033 raft_consensus.cc:348] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.572880 9188 raft_consensus.cc:2781] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:36:01.573096 6033 raft_consensus.cc:374] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.573184 9188 raft_consensus.cc:482] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 98dd96f425fc4ab3a6a54f2dfaaf9fab)
I20260808 13:36:01.573354 6033 raft_consensus.cc:729] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.573488 9188 raft_consensus.cc:3037] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:36:01.574112 6033 consensus_queue.cc:260] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.577100 6033 ts_tablet_manager.cc:1428] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260808 13:36:01.578948 9175 raft_consensus.cc:482] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.579537 9175 raft_consensus.cc:504] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.583916 9245 raft_consensus.cc:482] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.584504 9245 raft_consensus.cc:504] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.586426 9175 leader_election.cc:288] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:01.587906 9245 leader_election.cc:288] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:01.588340 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342027968b884f1cbc78b1371318dfc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:36:01.589684 5771 leader_election.cc:341] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.607380 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f53e5b5c4bd4a9385ad198b283c6e79" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:01.608423 5638 raft_consensus.cc:2379] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:36:01.609910 6026 tablet_bootstrap.cc:654] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.627483 6033 tablet_bootstrap.cc:492] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:01.636773 6033 tablet_bootstrap.cc:654] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:36:01.638278 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 178725us
I20260808 13:36:01.640686 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa493b8652e149f988d41bee811f6a2f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:01.644853 9188 raft_consensus.cc:504] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.646567 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" candidate_term: 6 candidate_status { last_received { term: 4 index: 597 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5"
I20260808 13:36:01.647115 7741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 5 FOLLOWER]: Advancing to term 6
I20260808 13:36:01.657128 7741 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63c875fc5bcd41a9b1bcf3bd4ae321d7 in term 6.
I20260808 13:36:01.662046 6046 tablet_bootstrap.cc:492] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:36:01.670100 6046 tablet_bootstrap.cc:654] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.674783 6026 tablet_bootstrap.cc:492] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:01.677681 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f53e5b5c4bd4a9385ad198b283c6e79" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:36:01.678424 5875 raft_consensus.cc:2379] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:36:01.679891 6026 ts_tablet_manager.cc:1397] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.390s user 0.010s sys 0.007s
I20260808 13:36:01.682749 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67327badb2524a2c817045a893bcad6e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:01.704813 6026 raft_consensus.cc:348] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.708884 6026 raft_consensus.cc:374] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.709195 6026 raft_consensus.cc:729] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.713414 6026 consensus_queue.cc:260] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.714855 6026 ts_tablet_manager.cc:1428] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.035s user 0.006s sys 0.002s
I20260808 13:36:01.722051 9188 leader_election.cc:288] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Requested vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:01.723500 9273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:01.724553 6033 tablet_bootstrap.cc:492] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:01.725011 6033 ts_tablet_manager.cc:1397] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.098s user 0.004s sys 0.010s
I20260808 13:36:01.726450 6026 tablet_bootstrap.cc:492] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:01.642983 9251 leader_election.cc:288] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [CANDIDATE]: Term 6 election: Requested vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:36:01.738010 6046 tablet_bootstrap.cc:492] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:36:01.738443 6046 ts_tablet_manager.cc:1397] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.077s user 0.008s sys 0.004s
I20260808 13:36:01.741827 6046 raft_consensus.cc:348] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.746680 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
W20260808 13:36:01.747030 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 56195.3us
I20260808 13:36:01.747644 5635 raft_consensus.cc:3037] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:36:01.748225 9029 consensus_queue.cc:1035] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:36:01.752239 6033 raft_consensus.cc:348] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.752908 6033 raft_consensus.cc:374] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.753157 6033 raft_consensus.cc:729] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.753731 6033 consensus_queue.cc:260] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.742486 6046 raft_consensus.cc:374] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.755338 6046 raft_consensus.cc:729] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.756004 6046 consensus_queue.cc:260] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.759497 6046 ts_tablet_manager.cc:1428] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20260808 13:36:01.760489 6033 ts_tablet_manager.cc:1428] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.035s user 0.006s sys 0.000s
W20260808 13:36:01.761497 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 82466us
I20260808 13:36:01.765107 9177 consensus_queue.cc:1035] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:36:01.766223 6033 tablet_bootstrap.cc:492] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:01.767299 5772 leader_election.cc:302] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:36:01.768029 9177 raft_consensus.cc:2781] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:01.768875 9177 raft_consensus.cc:686] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:01.770068 9177 consensus_queue.cc:237] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:36:01.778577 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 428305us
I20260808 13:36:01.782310 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa493b8652e149f988d41bee811f6a2f" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:36:01.784669 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67327badb2524a2c817045a893bcad6e" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:36:01.799443 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342027968b884f1cbc78b1371318dfc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:01.800056 5644 raft_consensus.cc:2454] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:01.814152 9269 raft_consensus.cc:482] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.814788 9269 raft_consensus.cc:504] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.817515 9269 leader_election.cc:288] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:01.818572 9269 raft_consensus.cc:482] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.819271 9269 raft_consensus.cc:504] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.822932 9278 raft_consensus.cc:482] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.823642 9278 raft_consensus.cc:504] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
W20260808 13:36:01.827644 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 60426.6us
I20260808 13:36:01.826431 9278 leader_election.cc:288] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
W20260808 13:36:01.832973 5550 leader_election.cc:341] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.833393 5550 leader_election.cc:302] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:01.834112 5283 leader_election.cc:302] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:01.834993 6026 tablet_bootstrap.cc:654] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.835335 9175 raft_consensus.cc:3037] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:01.822427 9269 leader_election.cc:288] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:01.835629 5635 raft_consensus.cc:2454] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946c9d8462644deba5b45fa724f1923c in term 2.
I20260808 13:36:01.837337 5285 leader_election.cc:302] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 946c9d8462644deba5b45fa724f1923c; no voters:
I20260808 13:36:01.838168 9188 raft_consensus.cc:2781] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:36:01.839033 9188 raft_consensus.cc:686] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [term 2 LEADER]: Becoming Leader. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Running, Role: LEADER
W20260808 13:36:01.839953 5551 leader_election.cc:341] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.840173 9188 consensus_queue.cc:237] T 42218e4330d147f389717108bfe0acea P 946c9d8462644deba5b45fa724f1923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:01.841437 6033 tablet_bootstrap.cc:654] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
W20260808 13:36:01.847162 5550 leader_election.cc:341] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.847555 5550 leader_election.cc:302] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:01.856488 9266 raft_consensus.cc:2726] T a11a2666f0b44d70a08a9195e8fcf69d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:01.860955 5551 leader_election.cc:341] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.862758 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604d2113451642329b877f8beff71618" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:01.864452 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dee4b3b14264f3aad0896f629ab5b27" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:01.874742 9269 raft_consensus.cc:2726] T c2c50fd06b8841c8b5a944401b495d2e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:01.876261 5550 leader_election.cc:341] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.879449 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b421ff1f094d8689010b4ed6c3f7af" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:01.880838 5550 leader_election.cc:302] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:01.882417 4864 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 6 FOLLOWER]: Refusing update from remote peer 63c875fc5bcd41a9b1bcf3bd4ae321d7: Log matching property violated. Preceding OpId in replica: term: 4 index: 595. Preceding OpId from leader: term: 6 index: 598. (index mismatch)
I20260808 13:36:01.884075 9273 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 598, Last known committed idx: 595, Time since last communication: 0.000s
I20260808 13:36:01.886965 9266 raft_consensus.cc:2726] T aa493b8652e149f988d41bee811f6a2f P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:01.888031 5550 leader_election.cc:341] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.888501 5550 leader_election.cc:302] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:01.888684 9245 raft_consensus.cc:482] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.889213 9266 raft_consensus.cc:2726] T 67327badb2524a2c817045a893bcad6e P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:01.882995 5551 leader_election.cc:341] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.889289 9245 raft_consensus.cc:504] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:36:01.889981 5551 leader_election.cc:341] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:36:01.890547 5551 leader_election.cc:341] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.883627 9269 raft_consensus.cc:482] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:01.891134 9269 raft_consensus.cc:504] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.891834 9245 leader_election.cc:288] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:01.893030 9269 leader_election.cc:288] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:01.893596 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2be108f26bb48d49216c6b7eef6b9b8" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
W20260808 13:36:01.895000 5771 leader_election.cc:341] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.906680 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4747a38c520d40848f4d3c261499aa88" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:01.909102 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dee4b3b14264f3aad0896f629ab5b27" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:36:01.909659 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42218e4330d147f389717108bfe0acea" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab"
I20260808 13:36:01.909710 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b421ff1f094d8689010b4ed6c3f7af" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:36:01.910676 6046 tablet_bootstrap.cc:492] T 6e70a5cf88d94b36913a6d1ccfcb979a P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:36:01.914949 9175 raft_consensus.cc:2726] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:01.915961 5866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604d2113451642329b877f8beff71618" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:36:01.919638 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4747a38c520d40848f4d3c261499aa88" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
I20260808 13:36:01.925338 6046 tablet_bootstrap.cc:654] T 6e70a5cf88d94b36913a6d1ccfcb979a P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:01.934271 5772 leader_election.cc:302] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
W20260808 13:36:01.935607 5550 leader_election.cc:341] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.937999 5638 raft_consensus.cc:1226] T 42218e4330d147f389717108bfe0acea P 5cd987c3dbdb4168beb73cd7e577c7cb [term 2 FOLLOWER]: Rejecting Update request from peer 98dd96f425fc4ab3a6a54f2dfaaf9fab for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260808 13:36:01.938992 5550 leader_election.cc:302] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
W20260808 13:36:01.939283 5551 leader_election.cc:341] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:36:01.941059 5550 leader_election.cc:341] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.941411 9245 raft_consensus.cc:2781] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:01.941762 9245 raft_consensus.cc:482] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:01.942070 9245 raft_consensus.cc:3037] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:01.943902 5550 leader_election.cc:302] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:01.948975 9245 raft_consensus.cc:504] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.956805 6033 tablet_bootstrap.cc:492] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:01.957805 6033 ts_tablet_manager.cc:1397] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.192s user 0.005s sys 0.011s
I20260808 13:36:01.965425 6033 raft_consensus.cc:348] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.970257 9266 raft_consensus.cc:2726] T 9dee4b3b14264f3aad0896f629ab5b27 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:01.973222 6046 tablet_bootstrap.cc:492] T 6e70a5cf88d94b36913a6d1ccfcb979a P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
I20260808 13:36:01.973656 6046 ts_tablet_manager.cc:1397] T 6e70a5cf88d94b36913a6d1ccfcb979a P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.063s user 0.007s sys 0.003s
I20260808 13:36:01.974421 9269 raft_consensus.cc:2726] T b6b421ff1f094d8689010b4ed6c3f7af P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:01.979192 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2be108f26bb48d49216c6b7eef6b9b8" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:01.979159 6046 raft_consensus.cc:348] T 6e70a5cf88d94b36913a6d1ccfcb979a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.979777 5636 raft_consensus.cc:2454] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:01.979843 6046 raft_consensus.cc:374] T 6e70a5cf88d94b36913a6d1ccfcb979a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:01.980083 6046 raft_consensus.cc:729] T 6e70a5cf88d94b36913a6d1ccfcb979a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:36:01.980715 6046 consensus_queue.cc:260] T 6e70a5cf88d94b36913a6d1ccfcb979a P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:01.982316 6046 ts_tablet_manager.cc:1428] T 6e70a5cf88d94b36913a6d1ccfcb979a P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
W20260808 13:36:01.985458 5550 leader_election.cc:341] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.985829 5550 leader_election.cc:302] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:01.992226 9266 raft_consensus.cc:2726] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:01.994993 5550 leader_election.cc:341] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:01.995371 5550 leader_election.cc:302] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:01.996258 9266 raft_consensus.cc:2726] T 4747a38c520d40848f4d3c261499aa88 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:02.000893 9188 raft_consensus.cc:482] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.001571 9188 raft_consensus.cc:504] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:02.004562 9188 leader_election.cc:288] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:02.004814 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d12106f0d794a79935999a2b8dbe7e9" candidate_uuid: "946c9d8462644deba5b45fa724f1923c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:02.005415 9266 raft_consensus.cc:482] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.005545 5636 raft_consensus.cc:2379] T 8d12106f0d794a79935999a2b8dbe7e9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 946c9d8462644deba5b45fa724f1923c in current term 1: Already voted for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in this term.
I20260808 13:36:02.004791 9303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c875fc5bcd41a9b1bcf3bd4ae321d7. Latest consensus state: current_term: 6 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:36:02.006894 9303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:02.006075 9266 raft_consensus.cc:504] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:02.008715 9266 leader_election.cc:288] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:02.009527 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342027968b884f1cbc78b1371318dfc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:36:02.009869 7741 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 6 FOLLOWER]: Refusing update from remote peer 63c875fc5bcd41a9b1bcf3bd4ae321d7: Log matching property violated. Preceding OpId in replica: term: 4 index: 596. Preceding OpId from leader: term: 6 index: 598. (index mismatch)
I20260808 13:36:02.010070 5379 raft_consensus.cc:3037] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.025215 6026 tablet_bootstrap.cc:492] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:02.025650 6026 ts_tablet_manager.cc:1397] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.299s user 0.012s sys 0.013s
I20260808 13:36:02.030722 6026 raft_consensus.cc:348] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.031584 6026 raft_consensus.cc:374] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:02.031859 6026 raft_consensus.cc:729] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:02.032533 6026 consensus_queue.cc:260] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.033644 5376 raft_consensus.cc:3037] T b0f778e7fa51414cb209fbe1aea92739 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.041169 5379 raft_consensus.cc:2454] T 342027968b884f1cbc78b1371318dfc9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:02.052438 5377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "290081bbe4164acd8808041b76441f0d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:02.003696 9245 leader_election.cc:288] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:02.015686 9029 consensus_queue.cc:1046] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:02.058789 6033 raft_consensus.cc:374] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:02.059085 6033 raft_consensus.cc:729] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:02.059286 9029 raft_consensus.cc:3032] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Stepping down as leader of term 1
I20260808 13:36:02.059617 9029 raft_consensus.cc:729] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:02.059751 6033 consensus_queue.cc:260] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.060330 9029 consensus_queue.cc:260] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } }
I20260808 13:36:02.061504 9029 raft_consensus.cc:3037] T 42218e4330d147f389717108bfe0acea P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:36:02.062048 6033 ts_tablet_manager.cc:1428] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.103s user 0.003s sys 0.002s
I20260808 13:36:02.016673 8929 consensus_queue.cc:1035] T 3424ca753bcc4f448d314c56aba3323e P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:02.063460 6033 tablet_bootstrap.cc:492] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:36:02.017457 5772 consensus_peers.cc:487] T 5c0b76c4fdd94a6e9d9bb7709ff1b4be P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): Couldn't send request to peer 5cd987c3dbdb4168beb73cd7e577c7cb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5c0b76c4fdd94a6e9d9bb7709ff1b4be. This is attempt 6: this message will repeat every 5th retry.
I20260808 13:36:02.022500 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "290081bbe4164acd8808041b76441f0d" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:02.063911 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 46552.6us
I20260808 13:36:02.042533 5771 leader_election.cc:302] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:02.044000 9269 raft_consensus.cc:482] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.064669 5772 leader_election.cc:302] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:36:02.064949 9269 raft_consensus.cc:504] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.065598 9273 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 598, Last known committed idx: 595, Time since last communication: 0.000s
I20260808 13:36:02.045493 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342027968b884f1cbc78b1371318dfc9" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:02.066823 5636 raft_consensus.cc:3037] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.066859 9269 leader_election.cc:288] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
W20260808 13:36:02.068082 5283 leader_election.cc:341] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Not found: Tablet not found: 8d12106f0d794a79935999a2b8dbe7e9
I20260808 13:36:02.068325 9129 raft_consensus.cc:2781] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:02.068445 5283 leader_election.cc:302] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c; no voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:02.068627 9129 raft_consensus.cc:482] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:02.069064 9129 raft_consensus.cc:3037] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.070984 9175 raft_consensus.cc:3037] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.079494 9245 raft_consensus.cc:2781] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:02.080057 9245 raft_consensus.cc:686] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:02.080804 9245 consensus_queue.cc:237] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.081782 9175 raft_consensus.cc:2726] T 8d12106f0d794a79935999a2b8dbe7e9 P 946c9d8462644deba5b45fa724f1923c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:02.096122 5636 raft_consensus.cc:2454] T 342027968b884f1cbc78b1371318dfc9 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:02.105933 6046 tablet_bootstrap.cc:492] T d2be108f26bb48d49216c6b7eef6b9b8 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:36:02.101507 6026 ts_tablet_manager.cc:1428] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.076s user 0.004s sys 0.004s
I20260808 13:36:02.120985 6026 tablet_bootstrap.cc:492] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
W20260808 13:36:02.063115 5551 leader_election.cc:341] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:36:02.122829 5550 leader_election.cc:341] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:02.123533 6046 tablet_bootstrap.cc:654] T d2be108f26bb48d49216c6b7eef6b9b8 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:02.124493 5550 leader_election.cc:302] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:02.125216 9269 raft_consensus.cc:2726] T 290081bbe4164acd8808041b76441f0d P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:02.125815 5550 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:34173, user_credentials={real_user=slave}} blocked reactor thread for 61165.5us
I20260808 13:36:02.127383 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "315c8b665cc9490da0b10623b9963154" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:02.129104 5550 leader_election.cc:341] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:36:02.131012 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 67760.5us
I20260808 13:36:02.115926 9129 raft_consensus.cc:504] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.134852 9129 leader_election.cc:288] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:02.176620 6033 tablet_bootstrap.cc:654] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:02.179366 5378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2be108f26bb48d49216c6b7eef6b9b8" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c"
I20260808 13:36:02.186075 9245 consensus_queue.cc:1035] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:02.190507 4965 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7: Sending DeleteTablet for 3 replicas of tablet 433fcbc6cdca458b8404ae5862f280a3
I20260808 13:36:02.192656 4965 catalog_manager.cc:6565] ReplaceTablet: tablet 433fcbc6cdca458b8404ae5862f280a3 deleted and replaced by tablet 2fddb48a2298484cb5d1830dea9ea20b
I20260808 13:36:02.193080 4965 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:49444 (request call id 47) took 2592 ms. Trace:
W20260808 13:36:02.196729 4961 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet ffbdf03ebe4e48a98450e90a3a78181c (table default.Table_510c442cafba44a9bf48a292ca01fbd4 [id=212e1314b3c04c4d8dd06d5730fa25cf], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet ffbdf03ebe4e48a98450e90a3a78181c consensus state has no leader
W20260808 13:36:02.197831 4961 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.4.170.2:45445 (request call id 299) took 2201 ms (client timeout 1000 ms). Trace:
I20260808 13:36:02.201380 5831 tablet_service.cc:1514] Processing DeleteTablet for tablet 433fcbc6cdca458b8404ae5862f280a3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2fddb48a2298484cb5d1830dea9ea20b) from {username='slave'} at 127.0.0.1:48546
I20260808 13:36:02.202461 5335 tablet_service.cc:1514] Processing DeleteTablet for tablet 433fcbc6cdca458b8404ae5862f280a3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2fddb48a2298484cb5d1830dea9ea20b) from {username='slave'} at 127.0.0.1:38994
I20260808 13:36:02.209394 6026 tablet_bootstrap.cc:654] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
W20260808 13:36:02.214102 5771 leader_election.cc:341] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:02.215993 9303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:36:02.216825 9303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:02.193338 4965 rpcz_store.cc:276] 0808 13:35:59.600842 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:59.615949 (+ 15107us) service_pool.cc:227] Handling call
0808 13:36:02.192995 (+2577046us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:36:00.464471 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:36:00.465115 (+ 644us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:36:00.465152 (+ 37us) write_op.cc:435] Acquired schema lock
0808 13:36:00.465164 (+ 12us) tablet.cc:660] Decoding operations
0808 13:36:00.465705 (+ 541us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:36:00.465781 (+ 76us) write_op.cc:641] Partition lock acquired for write op
0808 13:36:00.465814 (+ 33us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:36:00.466108 (+ 294us) tablet.cc:699] Row locks acquired
0808 13:36:00.466134 (+ 26us) write_op.cc:260] PREPARE: finished
0808 13:36:00.466341 (+ 207us) write_op.cc:270] Start()
0808 13:36:00.466527 (+ 186us) write_op.cc:276] Timestamp: P: 1786196160466245 usec, L: 0
0808 13:36:00.466580 (+ 53us) op_driver.cc:331] REPLICATION: starting
0808 13:36:00.470118 (+ 3538us) log.cc:842] Serialized 622 byte log entry
0808 13:36:02.165842 (+1695724us) op_driver.cc:444] REPLICATION: finished
0808 13:36:02.186777 (+ 20935us) write_op.cc:301] APPLY: starting
0808 13:36:02.186838 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0808 13:36:02.186973 (+ 135us) tablet.cc:1368] finished BulkCheckPresence
0808 13:36:02.187081 (+ 108us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:36:02.187530 (+ 449us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:36:02.187912 (+ 382us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:36:02.187971 (+ 59us) write_op.cc:312] APPLY: finished
0808 13:36:02.188370 (+ 399us) log.cc:842] Serialized 29 byte log entry
0808 13:36:02.189592 (+ 1222us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:36:02.189660 (+ 68us) write_op.cc:454] Released schema lock
0808 13:36:02.189698 (+ 38us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"dns_us":178,"child_traces":[["op",{"apply.queue_time_us":20733,"apply.run_cpu_time_us":1995,"apply.run_wall_time_us":3230,"num_ops":2,"prepare.queue_time_us":108864,"prepare.run_cpu_time_us":3431,"prepare.run_wall_time_us":17856,"replication_time_us":1699177,"thread_start_us":43138,"threads_started":1}]]}
I20260808 13:36:02.220060 6046 tablet_bootstrap.cc:492] T d2be108f26bb48d49216c6b7eef6b9b8 P 946c9d8462644deba5b45fa724f1923c: No bootstrap required, opened a new log
W20260808 13:36:02.198123 4961 rpcz_store.cc:269] 0808 13:35:59.995980 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:35:59.996088 (+ 108us) service_pool.cc:227] Handling call
0808 13:36:02.197723 (+2201635us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:36:00.364535 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:36:00.365067 (+ 532us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:36:00.365103 (+ 36us) write_op.cc:435] Acquired schema lock
0808 13:36:00.365130 (+ 27us) tablet.cc:660] Decoding operations
0808 13:36:00.365551 (+ 421us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:36:00.365627 (+ 76us) write_op.cc:641] Partition lock acquired for write op
0808 13:36:00.365659 (+ 32us) tablet.cc:683] Acquiring locks for 1 operations
0808 13:36:00.365887 (+ 228us) tablet.cc:699] Row locks acquired
0808 13:36:00.365915 (+ 28us) write_op.cc:260] PREPARE: finished
0808 13:36:00.366066 (+ 151us) write_op.cc:270] Start()
0808 13:36:00.366169 (+ 103us) write_op.cc:276] Timestamp: P: 1786196160366011 usec, L: 0
0808 13:36:00.366214 (+ 45us) op_driver.cc:331] REPLICATION: starting
0808 13:36:00.368925 (+ 2711us) log.cc:842] Serialized 527 byte log entry
0808 13:36:00.529249 (+160324us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b5400012320. stack: 00007f04bb18e033 00007f04bb18df30 00007f04bad0f534 00000000004db2b4 00000000004db249 00007f04c209a471 00007f04c20a3771 00007f04c20ae220 00007f04c20ae1c8 00007f04c20ae148 00007f04c20ae103 00007f04c20ad38c 00007f04c4098206 00007f04c4090808 00007f04c20ea38b 00007f04c20ece80
0808 13:36:02.068499 (+1539250us) op_driver.cc:444] REPLICATION: finished
0808 13:36:02.163076 (+ 94577us) write_op.cc:301] APPLY: starting
0808 13:36:02.163156 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0808 13:36:02.163297 (+ 141us) tablet.cc:1368] finished BulkCheckPresence
0808 13:36:02.163325 (+ 28us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:36:02.190161 (+ 26836us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:36:02.190306 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0808 13:36:02.190351 (+ 45us) write_op.cc:312] APPLY: finished
0808 13:36:02.190533 (+ 182us) log.cc:842] Serialized 23 byte log entry
0808 13:36:02.190724 (+ 191us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:36:02.190786 (+ 62us) write_op.cc:454] Released schema lock
0808 13:36:02.190815 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":94359,"apply.run_cpu_time_us":1394,"apply.run_wall_time_us":27975,"mutex_wait_us":1110,"num_ops":1,"prepare.queue_time_us":28929,"prepare.run_cpu_time_us":4097,"prepare.run_wall_time_us":99414,"raft.queue_time_us":1473,"raft.run_cpu_time_us":2634,"raft.run_wall_time_us":2633,"replication_time_us":1702193,"spinlock_wait_cycles":50777856,"thread_start_us":212303,"threads_started":3,"wal-append.queue_time_us":69019}]]}
I20260808 13:36:02.220590 6046 ts_tablet_manager.cc:1397] T d2be108f26bb48d49216c6b7eef6b9b8 P 946c9d8462644deba5b45fa724f1923c: Time spent bootstrapping tablet: real 0.115s user 0.010s sys 0.019s
I20260808 13:36:02.233326 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "315c8b665cc9490da0b10623b9963154" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:02.235553 9312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c875fc5bcd41a9b1bcf3bd4ae321d7. Latest consensus state: current_term: 6 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
W20260808 13:36:02.244422 4941 catalog_manager.cc:4821] TS 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): delete failed for tablet 433fcbc6cdca458b8404ae5862f280a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 433fcbc6cdca458b8404ae5862f280a3 already in progress: creating tablet
I20260808 13:36:02.262403 6026 tablet_bootstrap.cc:492] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:02.262876 6026 ts_tablet_manager.cc:1397] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.142s user 0.009s sys 0.002s
I20260808 13:36:02.263845 9320 raft_consensus.cc:482] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.264367 9320 raft_consensus.cc:504] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.265161 6026 raft_consensus.cc:348] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.265807 6026 raft_consensus.cc:374] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:02.266062 6026 raft_consensus.cc:729] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:02.266543 9319 raft_consensus.cc:482] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.267287 9320 leader_election.cc:288] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:02.267035 9319 raft_consensus.cc:504] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.269786 9319 leader_election.cc:288] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:02.271713 9273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:36:02.272668 9273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:02.273602 9273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:36:02.274355 9273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c875fc5bcd41a9b1bcf3bd4ae321d7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:02.274178 9312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:02.237107 6046 raft_consensus.cc:348] T d2be108f26bb48d49216c6b7eef6b9b8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.275771 6046 raft_consensus.cc:374] T d2be108f26bb48d49216c6b7eef6b9b8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:02.276063 6046 raft_consensus.cc:729] T d2be108f26bb48d49216c6b7eef6b9b8 P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c9d8462644deba5b45fa724f1923c, State: Initialized, Role: FOLLOWER
I20260808 13:36:02.276687 6046 consensus_queue.cc:260] T d2be108f26bb48d49216c6b7eef6b9b8 P 946c9d8462644deba5b45fa724f1923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.280797 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2be108f26bb48d49216c6b7eef6b9b8" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:02.281391 5637 raft_consensus.cc:3037] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.278910 6046 ts_tablet_manager.cc:1428] T d2be108f26bb48d49216c6b7eef6b9b8 P 946c9d8462644deba5b45fa724f1923c: Time spent starting tablet: real 0.050s user 0.005s sys 0.001s
I20260808 13:36:02.289557 6046 tablet_bootstrap.cc:492] T 604d2113451642329b877f8beff71618 P 946c9d8462644deba5b45fa724f1923c: Bootstrap starting.
I20260808 13:36:02.294123 5637 raft_consensus.cc:2454] T d2be108f26bb48d49216c6b7eef6b9b8 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:02.301136 6046 tablet_bootstrap.cc:654] T 604d2113451642329b877f8beff71618 P 946c9d8462644deba5b45fa724f1923c: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:02.304682 5379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70a5cf88d94b36913a6d1ccfcb979a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:02.305493 5379 raft_consensus.cc:2454] T 6e70a5cf88d94b36913a6d1ccfcb979a P 946c9d8462644deba5b45fa724f1923c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
W20260808 13:36:02.308871 4942 catalog_manager.cc:4821] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): delete failed for tablet 433fcbc6cdca458b8404ae5862f280a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 433fcbc6cdca458b8404ae5862f280a3 already in progress: creating tablet
I20260808 13:36:02.315810 5771 leader_election.cc:302] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:02.317296 9177 raft_consensus.cc:2781] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:02.317806 9177 raft_consensus.cc:482] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:02.318358 9177 raft_consensus.cc:3037] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.323642 5376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604d2113451642329b877f8beff71618" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c9d8462644deba5b45fa724f1923c" is_pre_election: true
I20260808 13:36:02.327807 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70a5cf88d94b36913a6d1ccfcb979a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:02.328424 5644 raft_consensus.cc:2454] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:02.328285 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604d2113451642329b877f8beff71618" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:02.328972 5629 raft_consensus.cc:2454] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:02.333127 4776 master-stress-test.cc:580] Killed test thread 9/10
I20260808 13:36:02.333487 4776 master-stress-test.cc:578] Killing test thread 10/10
I20260808 13:36:02.333798 4776 master-stress-test.cc:580] Killed test thread 10/10
I20260808 13:36:02.334147 4776 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu with pid 5241
I20260808 13:36:02.337888 9266 raft_consensus.cc:482] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.338312 9266 raft_consensus.cc:504] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:36:02.340140 9266 leader_election.cc:288] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:02.343528 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293e9540498e420d8ec873472d0d29fc" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:02.344930 5550 leader_election.cc:341] T 293e9540498e420d8ec873472d0d29fc P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:02.345830 9319 raft_consensus.cc:482] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.346298 9319 raft_consensus.cc:504] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.348601 9319 leader_election.cc:288] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
W20260808 13:36:02.351979 4942 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ffbdf03ebe4e48a98450e90a3a78181c (table default.Table_510c442cafba44a9bf48a292ca01fbd4 [id=212e1314b3c04c4d8dd06d5730fa25cf], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ffbdf03ebe4e48a98450e90a3a78181c consensus state has no leader
I20260808 13:36:02.354660 9312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } } }
I20260808 13:36:02.356143 9312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:02.356515 6033 tablet_bootstrap.cc:492] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:02.356950 6033 ts_tablet_manager.cc:1397] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.294s user 0.010s sys 0.010s
I20260808 13:36:02.359239 9320 consensus_queue.cc:1035] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:02.359251 6033 raft_consensus.cc:348] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.362320 6033 raft_consensus.cc:374] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:02.362571 6033 raft_consensus.cc:729] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:02.363261 6033 consensus_queue.cc:260] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.365126 6033 ts_tablet_manager.cc:1428] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 13:36:02.370893 6026 consensus_queue.cc:260] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.376310 5772 leader_election.cc:302] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters: 946c9d8462644deba5b45fa724f1923c
I20260808 13:36:02.377308 9319 raft_consensus.cc:2781] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:02.379403 9319 raft_consensus.cc:686] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:02.380311 9319 consensus_queue.cc:237] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.392163 6033 tablet_bootstrap.cc:492] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:02.395439 6026 ts_tablet_manager.cc:1428] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.132s user 0.002s sys 0.003s
I20260808 13:36:02.396346 6026 tablet_bootstrap.cc:492] T 0329c348c88a4ddda0f2b90be7a4af76 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:02.403730 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet 433fcbc6cdca458b8404ae5862f280a3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2fddb48a2298484cb5d1830dea9ea20b) from {username='slave'} at 127.0.0.1:48944
I20260808 13:36:02.405761 5592 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f0a6d9bd2c74d0790badd7977f027e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:34 UTC) from {username='slave'} at 127.0.0.1:48944
I20260808 13:36:02.406400 5598 tablet_service.cc:1514] Processing DeleteTablet for tablet 3451d72548a140c68a6ccc2b566a09b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:32 UTC) from {username='slave'} at 127.0.0.1:48944
I20260808 13:36:02.408152 5598 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea8f5322a404837b91e1d96749926b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:35:30 UTC) from {username='slave'} at 127.0.0.1:48944
W20260808 13:36:02.415764 5551 leader_election.cc:341] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:02.416174 5551 leader_election.cc:302] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb; no voters: 946c9d8462644deba5b45fa724f1923c, 98dd96f425fc4ab3a6a54f2dfaaf9fab
I20260808 13:36:02.410459 5772 leader_election.cc:302] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:02.417119 9129 raft_consensus.cc:2781] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:02.417465 9129 raft_consensus.cc:482] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:02.417752 9129 raft_consensus.cc:3037] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.434478 6033 tablet_bootstrap.cc:654] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:02.440491 6026 tablet_bootstrap.cc:654] T 0329c348c88a4ddda0f2b90be7a4af76 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:02.449450 9177 raft_consensus.cc:504] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.457051 9344 tablet_replica.cc:330] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:36:02.459024 9266 raft_consensus.cc:2726] T 315c8b665cc9490da0b10623b9963154 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:02.460726 4943 catalog_manager.cc:4821] TS 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577): delete failed for tablet 433fcbc6cdca458b8404ae5862f280a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 433fcbc6cdca458b8404ae5862f280a3 already in progress: creating tablet
I20260808 13:36:02.468276 6033 tablet_bootstrap.cc:492] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:02.468803 6033 ts_tablet_manager.cc:1397] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.077s user 0.014s sys 0.000s
I20260808 13:36:02.471568 6033 raft_consensus.cc:348] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.472412 6033 raft_consensus.cc:374] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:02.472734 6033 raft_consensus.cc:729] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:02.473111 9129 raft_consensus.cc:504] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:36:02.485316 5551 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 69573us
I20260808 13:36:02.473304 6033 consensus_queue.cc:260] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.489846 6033 ts_tablet_manager.cc:1428] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20260808 13:36:02.492818 9344 raft_consensus.cc:2229] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:02.500955 9177 leader_election.cc:288] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:02.502112 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7f6487e28f4aaa94d448be71f8a3c2" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:02.522058 5629 raft_consensus.cc:2454] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:02.523114 9344 raft_consensus.cc:2258] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:02.518088 6033 tablet_bootstrap.cc:492] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:02.550720 9129 leader_election.cc:288] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:02.553279 9344 ts_tablet_manager.cc:1905] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:02.559900 5831 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab078df59e34e1296526175d74e38f7 (DEFAULT_TABLE table=default.Table_d39fd423849c45c69b7338e8c6a87c30 [id=f5bbd5d961794ddcab1f07f3931ab031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.561419 5831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab078df59e34e1296526175d74e38f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.563902 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab078df59e34e1296526175d74e38f7 (DEFAULT_TABLE table=default.Table_d39fd423849c45c69b7338e8c6a87c30 [id=f5bbd5d961794ddcab1f07f3931ab031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.563922 5827 tablet_service.cc:1467] Processing CreateTablet for tablet 2e307a9ea77148c4a74385ace4bfeb73 (DEFAULT_TABLE table=default.Table_c171ae1c3b234b4e8b6f3def21f27924 [id=efc0b7748878445c8245bcd3e9657abe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.564446 5826 tablet_service.cc:1467] Processing CreateTablet for tablet b49499111fdb45a48e98aa6d0894e2a0 (DEFAULT_TABLE table=default.Table_9d2535df26964377be89e6da0a54e8b4 [id=eb356c7ce7ca4cc1a5a51234f3404e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.565147 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab078df59e34e1296526175d74e38f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.565447 5827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e307a9ea77148c4a74385ace4bfeb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.566155 5826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b49499111fdb45a48e98aa6d0894e2a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.568507 5598 tablet_service.cc:1467] Processing CreateTablet for tablet 2e307a9ea77148c4a74385ace4bfeb73 (DEFAULT_TABLE table=default.Table_c171ae1c3b234b4e8b6f3def21f27924 [id=efc0b7748878445c8245bcd3e9657abe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.569692 5598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e307a9ea77148c4a74385ace4bfeb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.575914 5592 tablet_service.cc:1467] Processing CreateTablet for tablet b49499111fdb45a48e98aa6d0894e2a0 (DEFAULT_TABLE table=default.Table_9d2535df26964377be89e6da0a54e8b4 [id=eb356c7ce7ca4cc1a5a51234f3404e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.577206 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b49499111fdb45a48e98aa6d0894e2a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.579501 5842 tablet_service.cc:1467] Processing CreateTablet for tablet fbe6707a90fd45a2bfda4ec4af1bddd9 (DEFAULT_TABLE table=default.Table_f985e4cebca740fea14f259fcfacec86 [id=e0c4baae657c406b9d7c8c59fb77e10e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.580566 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbe6707a90fd45a2bfda4ec4af1bddd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.593667 5613 tablet_service.cc:1467] Processing CreateTablet for tablet fbe6707a90fd45a2bfda4ec4af1bddd9 (DEFAULT_TABLE table=default.Table_f985e4cebca740fea14f259fcfacec86 [id=e0c4baae657c406b9d7c8c59fb77e10e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.594877 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbe6707a90fd45a2bfda4ec4af1bddd9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:36:02.606753 5553 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.4.170.60:37135, user_credentials={real_user=slave}} blocked reactor thread for 64891.6us
I20260808 13:36:02.612387 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70a5cf88d94b36913a6d1ccfcb979a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:02.612649 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604d2113451642329b877f8beff71618" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:02.613018 5635 raft_consensus.cc:3037] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.613148 5636 raft_consensus.cc:3037] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.619321 5635 raft_consensus.cc:2454] T 6e70a5cf88d94b36913a6d1ccfcb979a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:02.620196 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 8f16ee6c8acb40d994347546fa0fdbe7 (DEFAULT_TABLE table=default.Table_a8ff8993913d4a05a3d8ab3bb339b7ac [id=8ac0944cbe564a949c732ec146a9cf94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.621417 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f16ee6c8acb40d994347546fa0fdbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.623799 5636 raft_consensus.cc:2454] T 604d2113451642329b877f8beff71618 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:02.625566 6033 tablet_bootstrap.cc:654] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:02.626147 6026 tablet_bootstrap.cc:492] T 0329c348c88a4ddda0f2b90be7a4af76 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:02.626580 6026 ts_tablet_manager.cc:1397] T 0329c348c88a4ddda0f2b90be7a4af76 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.230s user 0.011s sys 0.011s
I20260808 13:36:02.629599 5772 leader_election.cc:302] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:02.630841 5772 leader_election.cc:302] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:02.631726 5772 leader_election.cc:302] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:02.632813 9319 raft_consensus.cc:2781] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:02.633126 9319 raft_consensus.cc:482] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:02.633412 9319 raft_consensus.cc:3037] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.634212 8929 raft_consensus.cc:2781] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:02.634701 8929 raft_consensus.cc:686] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:02.635735 8929 consensus_queue.cc:237] T 6e70a5cf88d94b36913a6d1ccfcb979a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.639592 6026 raft_consensus.cc:348] T 0329c348c88a4ddda0f2b90be7a4af76 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.642970 6026 raft_consensus.cc:374] T 0329c348c88a4ddda0f2b90be7a4af76 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:02.644047 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 8f16ee6c8acb40d994347546fa0fdbe7 (DEFAULT_TABLE table=default.Table_a8ff8993913d4a05a3d8ab3bb339b7ac [id=8ac0944cbe564a949c732ec146a9cf94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.647927 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 9b427a762b934e988009184bac9ea674 (DEFAULT_TABLE table=default.Table_dbb6f93863704e4e9c573d989a367368 [id=2571402ea4b7478098e9596eb3235370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.654932 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 16a055fbb45844a89991d6972619eef4 (DEFAULT_TABLE table=default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.655247 6026 raft_consensus.cc:729] T 0329c348c88a4ddda0f2b90be7a4af76 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:02.656236 6026 consensus_queue.cc:260] T 0329c348c88a4ddda0f2b90be7a4af76 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.658545 6026 ts_tablet_manager.cc:1428] T 0329c348c88a4ddda0f2b90be7a4af76 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.032s user 0.003s sys 0.002s
I20260808 13:36:02.659722 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f16ee6c8acb40d994347546fa0fdbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.660856 6026 tablet_bootstrap.cc:492] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:02.663291 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16a055fbb45844a89991d6972619eef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.666159 6026 tablet_bootstrap.cc:654] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:02.667331 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b427a762b934e988009184bac9ea674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.671584 9320 raft_consensus.cc:2781] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:02.676321 9320 raft_consensus.cc:686] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:02.677224 9320 consensus_queue.cc:237] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:36:02.684821 4823 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 538d9a883e9d46668fde30876352d20e (table default.Table_41f3478fe8c94fbfaf7fcbdee22ce987 [id=e3e121c36ff44ea0a3a98185c8cce381], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 538d9a883e9d46668fde30876352d20e consensus state has no leader
I20260808 13:36:02.686522 6026 tablet_bootstrap.cc:492] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:02.687237 6026 ts_tablet_manager.cc:1397] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.000s
I20260808 13:36:02.688212 9319 raft_consensus.cc:504] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.690114 6026 raft_consensus.cc:348] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:36:02.695652 5772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.2:40577, user_credentials={real_user=slave}} blocked reactor thread for 63238us
I20260808 13:36:02.697914 6026 raft_consensus.cc:374] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:02.698302 6026 raft_consensus.cc:729] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:02.699137 6026 consensus_queue.cc:260] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.701000 5829 tablet_service.cc:1467] Processing CreateTablet for tablet 16a055fbb45844a89991d6972619eef4 (DEFAULT_TABLE table=default.Table_3728a7744c9c45beb1755624a26cd410 [id=b791a5dea71e4f7c80402859019c49be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.700981 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 9b427a762b934e988009184bac9ea674 (DEFAULT_TABLE table=default.Table_dbb6f93863704e4e9c573d989a367368 [id=2571402ea4b7478098e9596eb3235370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.703265 6026 ts_tablet_manager.cc:1428] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.016s user 0.004s sys 0.002s
I20260808 13:36:02.703969 9319 leader_election.cc:288] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:02.704592 6026 tablet_bootstrap.cc:492] T 8d06a8cdfabd4adaa4fef064ee6da7d3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:02.715574 9348 raft_consensus.cc:482] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.716005 9348 raft_consensus.cc:504] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.719766 9348 leader_election.cc:288] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:02.720014 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b2335c2182d4c2abaa254f4554fd473" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:02.721417 5550 leader_election.cc:341] T 7b2335c2182d4c2abaa254f4554fd473 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:02.722795 6026 tablet_bootstrap.cc:654] T 8d06a8cdfabd4adaa4fef064ee6da7d3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:02.733762 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16a055fbb45844a89991d6972619eef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.735152 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b427a762b934e988009184bac9ea674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.744776 5826 tablet_service.cc:1467] Processing CreateTablet for tablet 55ca543166ab479590ca4e38289024cd (DEFAULT_TABLE table=default.Table_6c7876f6c0ea460d8fa5cd00a7baee97 [id=2d102da12b5240a59c2ab0bcacb2c405]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.745091 9348 raft_consensus.cc:482] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.745921 5826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55ca543166ab479590ca4e38289024cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.745563 9348 raft_consensus.cc:504] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.747480 9348 leader_election.cc:288] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:02.749043 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab26fa1872e4acf874f6577bded5e43" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:02.750278 5550 leader_election.cc:341] T 3ab26fa1872e4acf874f6577bded5e43 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:02.753484 5598 tablet_service.cc:1467] Processing CreateTablet for tablet f5f26063029f49d2855428be49bb5a3b (DEFAULT_TABLE table=default.Table_efd769b05e1142088d56528e943072ed [id=31c20a71a6144b238a2da27b7ea21863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.756129 5598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5f26063029f49d2855428be49bb5a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.759577 5614 tablet_service.cc:1467] Processing CreateTablet for tablet 55ca543166ab479590ca4e38289024cd (DEFAULT_TABLE table=default.Table_6c7876f6c0ea460d8fa5cd00a7baee97 [id=2d102da12b5240a59c2ab0bcacb2c405]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.762594 5842 tablet_service.cc:1467] Processing CreateTablet for tablet f5f26063029f49d2855428be49bb5a3b (DEFAULT_TABLE table=default.Table_efd769b05e1142088d56528e943072ed [id=31c20a71a6144b238a2da27b7ea21863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.763674 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5f26063029f49d2855428be49bb5a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.766096 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55ca543166ab479590ca4e38289024cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.782143 6033 tablet_bootstrap.cc:492] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:02.789099 9360 tablet.cc:1722] T 37495b29698940edb2924eb612e94bd7 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:36:02.800046 5827 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8fda3bd1ba4669994c78e6942185f7 (DEFAULT_TABLE table=default.Table_5d7feb173847448393c48160e861f0c1 [id=308f2ec39c62416ab0d396aad04351fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.800336 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7f6487e28f4aaa94d448be71f8a3c2" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:02.801368 5827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c8fda3bd1ba4669994c78e6942185f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.801362 5629 raft_consensus.cc:3037] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.815892 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8fda3bd1ba4669994c78e6942185f7 (DEFAULT_TABLE table=default.Table_5d7feb173847448393c48160e861f0c1 [id=308f2ec39c62416ab0d396aad04351fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:02.817083 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c8fda3bd1ba4669994c78e6942185f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:02.840580 9348 raft_consensus.cc:482] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.841048 9348 raft_consensus.cc:504] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.844578 9348 leader_election.cc:288] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:02.841493 9029 consensus_queue.cc:1035] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260808 13:36:02.844410 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad151fca07264d9bae6e07ca781c8e04" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:02.850916 5550 leader_election.cc:341] T ad151fca07264d9bae6e07ca781c8e04 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:02.836597 9366 tablet.cc:1722] T 37495b29698940edb2924eb612e94bd7 P 5cd987c3dbdb4168beb73cd7e577c7cb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260808 13:36:02.861975 9291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:36:02.866358 9320 raft_consensus.cc:482] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.866952 9320 raft_consensus.cc:504] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.872179 9320 leader_election.cc:288] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:02.829815 6033 ts_tablet_manager.cc:1397] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.312s user 0.007s sys 0.004s
I20260808 13:36:02.889811 6033 raft_consensus.cc:348] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.922367 6033 raft_consensus.cc:374] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:02.905458 9269 raft_consensus.cc:482] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:02.924703 6033 raft_consensus.cc:729] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:02.927433 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61dfd47db0a74ed8b7228b559dd68459" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:02.928231 5635 raft_consensus.cc:2454] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:02.935272 5772 leader_election.cc:302] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:02.936934 9029 raft_consensus.cc:2781] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:02.937367 9029 raft_consensus.cc:482] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:02.937764 9029 raft_consensus.cc:3037] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:02.932431 6033 consensus_queue.cc:260] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.877451 5629 raft_consensus.cc:2454] T cb7f6487e28f4aaa94d448be71f8a3c2 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:02.953204 9269 raft_consensus.cc:504] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:36:02.963617 9029 raft_consensus.cc:504] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:02.965243 6026 tablet_bootstrap.cc:492] T 8d06a8cdfabd4adaa4fef064ee6da7d3 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:02.965669 9029 leader_election.cc:288] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:02.965862 6026 ts_tablet_manager.cc:1397] T 8d06a8cdfabd4adaa4fef064ee6da7d3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.261s user 0.041s sys 0.076s
I20260808 13:36:02.966639 6033 ts_tablet_manager.cc:1428] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.088s user 0.008s sys 0.000s
I20260808 13:36:02.996206 9269 leader_election.cc:288] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:03.011368 9368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:36:03.014822 9368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:36:03.017937 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e672d7386144b299688abea9c64854c" candidate_term: 7 candidate_status { last_received { term: 6 index: 599 } } ignore_live_leader: false dest_uuid: "87e8b888041b4135af55792050defcb5" is_pre_election: true
I20260808 13:36:03.019870 6026 raft_consensus.cc:348] T 8d06a8cdfabd4adaa4fef064ee6da7d3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.020547 6026 raft_consensus.cc:374] T 8d06a8cdfabd4adaa4fef064ee6da7d3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:03.020789 6026 raft_consensus.cc:729] T 8d06a8cdfabd4adaa4fef064ee6da7d3 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:03.025355 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1607745896141f68535b751dc211796" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:03.026948 5550 leader_election.cc:341] T c1607745896141f68535b751dc211796 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
W20260808 13:36:03.029075 5551 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.4.170.61:46555, user_credentials={real_user=slave}} blocked reactor thread for 69128.5us
I20260808 13:36:03.030737 9368 leader_election.cc:288] T 00000000000000000000000000000000 P 3e672d7386144b299688abea9c64854c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555), 87e8b888041b4135af55792050defcb5 (127.4.170.60:37135)
I20260808 13:36:03.031127 5772 leader_election.cc:302] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:03.032320 9245 raft_consensus.cc:2781] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:03.032778 9245 raft_consensus.cc:686] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:03.033536 9245 consensus_queue.cc:237] T cb7f6487e28f4aaa94d448be71f8a3c2 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.035759 6026 consensus_queue.cc:260] T 8d06a8cdfabd4adaa4fef064ee6da7d3 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.044087 6026 ts_tablet_manager.cc:1428] T 8d06a8cdfabd4adaa4fef064ee6da7d3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.078s user 0.005s sys 0.001s
I20260808 13:36:03.045002 6026 tablet_bootstrap.cc:492] T 01982f911d394aee8f4a3cd3b7110100 P 5cd987c3dbdb4168beb73cd7e577c7cb: Bootstrap starting.
I20260808 13:36:03.051403 6033 tablet_bootstrap.cc:492] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:03.058780 6033 tablet_bootstrap.cc:654] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:03.064586 9269 raft_consensus.cc:482] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:03.065068 9269 raft_consensus.cc:504] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.067267 9269 leader_election.cc:288] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173)
I20260808 13:36:03.069212 5869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b316390787f4170956f1c47a525d430" candidate_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" is_pre_election: true
W20260808 13:36:03.070729 5550 leader_election.cc:341] T 0b316390787f4170956f1c47a525d430 P 5cd987c3dbdb4168beb73cd7e577c7cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98dd96f425fc4ab3a6a54f2dfaaf9fab (127.4.170.3:34173): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:03.089318 5636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61dfd47db0a74ed8b7228b559dd68459" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb"
I20260808 13:36:03.089861 5636 raft_consensus.cc:3037] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:03.100528 5636 raft_consensus.cc:2454] T 61dfd47db0a74ed8b7228b559dd68459 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 1.
I20260808 13:36:03.130205 5772 leader_election.cc:302] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:03.131057 9353 raft_consensus.cc:2781] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:03.131522 9353 raft_consensus.cc:686] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 LEADER]: Becoming Leader. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Running, Role: LEADER
I20260808 13:36:03.132320 9353 consensus_queue.cc:237] T 61dfd47db0a74ed8b7228b559dd68459 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.165307 6033 tablet_bootstrap.cc:492] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:03.165735 6033 ts_tablet_manager.cc:1397] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.114s user 0.010s sys 0.001s
I20260808 13:36:03.181159 9177 consensus_queue.cc:1035] T 342027968b884f1cbc78b1371318dfc9 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:03.195398 9344 ts_tablet_manager.cc:1918] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:03.200984 9344 log.cc:1196] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/1f0a6d9bd2c74d0790badd7977f027e3
I20260808 13:36:03.211016 8929 consensus_queue.cc:1035] T d2be108f26bb48d49216c6b7eef6b9b8 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:03.200225 6033 raft_consensus.cc:348] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:36:03.207635 9344 ts_tablet_manager.cc:1939] T 1f0a6d9bd2c74d0790badd7977f027e3 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting consensus metadata
I20260808 13:36:03.220173 6033 raft_consensus.cc:374] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:03.221778 6033 raft_consensus.cc:729] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:03.223618 6033 consensus_queue.cc:260] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:36:03.245709 9344 tablet_replica.cc:330] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb: stopping tablet replica
I20260808 13:36:03.256001 6033 ts_tablet_manager.cc:1428] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.090s user 0.008s sys 0.000s
I20260808 13:36:03.267535 6026 tablet_bootstrap.cc:654] T 01982f911d394aee8f4a3cd3b7110100 P 5cd987c3dbdb4168beb73cd7e577c7cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:03.275035 9344 raft_consensus.cc:2229] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:03.275887 9344 raft_consensus.cc:2258] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:03.299330 6033 tablet_bootstrap.cc:492] T 3ab26fa1872e4acf874f6577bded5e43 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:03.306396 6026 tablet_bootstrap.cc:492] T 01982f911d394aee8f4a3cd3b7110100 P 5cd987c3dbdb4168beb73cd7e577c7cb: No bootstrap required, opened a new log
I20260808 13:36:03.307025 6026 ts_tablet_manager.cc:1397] T 01982f911d394aee8f4a3cd3b7110100 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent bootstrapping tablet: real 0.262s user 0.014s sys 0.001s
I20260808 13:36:03.309175 6026 raft_consensus.cc:348] T 01982f911d394aee8f4a3cd3b7110100 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.309944 6026 raft_consensus.cc:374] T 01982f911d394aee8f4a3cd3b7110100 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:03.310441 6026 raft_consensus.cc:729] T 01982f911d394aee8f4a3cd3b7110100 P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd987c3dbdb4168beb73cd7e577c7cb, State: Initialized, Role: FOLLOWER
I20260808 13:36:03.311666 6026 consensus_queue.cc:260] T 01982f911d394aee8f4a3cd3b7110100 P 5cd987c3dbdb4168beb73cd7e577c7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.316480 6033 tablet_bootstrap.cc:654] T 3ab26fa1872e4acf874f6577bded5e43 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:03.317884 9344 ts_tablet_manager.cc:1905] T 3451d72548a140c68a6ccc2b566a09b1 P 5cd987c3dbdb4168beb73cd7e577c7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 13:36:03.335036 7537 connection.cc:535] server connection from 127.4.170.1:49557 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 13:36:03.335912 8929 raft_consensus.cc:482] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:03.336349 8929 raft_consensus.cc:504] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.338025 8929 leader_election.cc:288] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:03.340193 5635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d2aae6405349fc887ec8527464e46a" candidate_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" is_pre_election: true
I20260808 13:36:03.340966 5635 raft_consensus.cc:2454] T e1d2aae6405349fc887ec8527464e46a P 5cd987c3dbdb4168beb73cd7e577c7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98dd96f425fc4ab3a6a54f2dfaaf9fab in term 0.
I20260808 13:36:03.343248 5772 leader_election.cc:302] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd987c3dbdb4168beb73cd7e577c7cb, 98dd96f425fc4ab3a6a54f2dfaaf9fab; no voters:
I20260808 13:36:03.344180 8929 raft_consensus.cc:2781] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:03.344561 8929 raft_consensus.cc:482] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:03.344807 8929 raft_consensus.cc:3037] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:03.348598 6026 ts_tablet_manager.cc:1428] T 01982f911d394aee8f4a3cd3b7110100 P 5cd987c3dbdb4168beb73cd7e577c7cb: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260808 13:36:03.352566 8929 raft_consensus.cc:504] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
W20260808 13:36:03.354315 4942 connection.cc:535] client connection to 127.4.170.1:40797 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:03.355165 4942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 7 similar messages]
W20260808 13:36:03.361593 5551 connection.cc:535] client connection to 127.4.170.1:40797 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:03.362063 5551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 13:36:03.364012 4776 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu with pid 5485
W20260808 13:36:03.361125 5771 connection.cc:535] client connection to 127.4.170.1:40797 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:03.374993 5771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 13:36:03.392089 6033 tablet_bootstrap.cc:492] T 3ab26fa1872e4acf874f6577bded5e43 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:03.392551 6033 ts_tablet_manager.cc:1397] T 3ab26fa1872e4acf874f6577bded5e43 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.093s user 0.009s sys 0.000s
W20260808 13:36:03.403349 4942 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 48125.1us
I20260808 13:36:03.404604 8929 leader_election.cc:288] T e1d2aae6405349fc887ec8527464e46a P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 election: Requested vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:03.406836 6033 raft_consensus.cc:348] T 3ab26fa1872e4acf874f6577bded5e43 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.407539 6033 raft_consensus.cc:374] T 3ab26fa1872e4acf874f6577bded5e43 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:03.407786 6033 raft_consensus.cc:729] T 3ab26fa1872e4acf874f6577bded5e43 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:03.408350 6033 consensus_queue.cc:260] T 3ab26fa1872e4acf874f6577bded5e43 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.410380 6033 ts_tablet_manager.cc:1428] T 3ab26fa1872e4acf874f6577bded5e43 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
W20260808 13:36:03.420800 4942 catalog_manager.cc:4584] TS 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): CreateTablet RPC failed for tablet 3ab078df59e34e1296526175d74e38f7: Network error: Client connection negotiation failed: client connection to 127.4.170.1:40797: connect: Connection refused (error 111)
I20260808 13:36:03.421443 6033 tablet_bootstrap.cc:492] T d439b27c3ad34619972c71d652f1ff10 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
I20260808 13:36:03.477108 9394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 63c875fc5bcd41a9b1bcf3bd4ae321d7)
I20260808 13:36:03.477726 9394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e672d7386144b299688abea9c64854c" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 34069 } } peers { permanent_uuid: "63c875fc5bcd41a9b1bcf3bd4ae321d7" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 46555 } } peers { permanent_uuid: "87e8b888041b4135af55792050defcb5" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 37135 } }
I20260808 13:36:03.479938 9394 leader_election.cc:288] T 00000000000000000000000000000000 P 87e8b888041b4135af55792050defcb5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3e672d7386144b299688abea9c64854c (127.4.170.62:34069), 63c875fc5bcd41a9b1bcf3bd4ae321d7 (127.4.170.61:46555)
I20260808 13:36:03.481999 4864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87e8b888041b4135af55792050defcb5" candidate_term: 7 candidate_status { last_received { term: 6 index: 599 } } ignore_live_leader: false dest_uuid: "3e672d7386144b299688abea9c64854c" is_pre_election: true
W20260808 13:36:03.525539 4820 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.170.61:46555 timed out after 0.508s (SENT) [suppressed 1 similar messages]
W20260808 13:36:03.547041 5771 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.1:40797, user_credentials={real_user=slave}} blocked reactor thread for 138956us
I20260808 13:36:03.534417 8929 raft_consensus.cc:482] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:03.552374 9353 raft_consensus.cc:482] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:03.553108 9353 raft_consensus.cc:504] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } }
I20260808 13:36:03.557582 6033 tablet_bootstrap.cc:654] T d439b27c3ad34619972c71d652f1ff10 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:03.549043 8929 raft_consensus.cc:504] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:36:03.559370 9353 leader_election.cc:288] T aa493b8652e149f988d41bee811f6a2f P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797), 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577)
I20260808 13:36:03.570535 8929 leader_election.cc:288] T 3451d72548a140c68a6ccc2b566a09b1 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd987c3dbdb4168beb73cd7e577c7cb (127.4.170.2:40577), 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797)
I20260808 13:36:03.597476 6033 tablet_bootstrap.cc:492] T d439b27c3ad34619972c71d652f1ff10 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: No bootstrap required, opened a new log
I20260808 13:36:03.599006 6033 ts_tablet_manager.cc:1397] T d439b27c3ad34619972c71d652f1ff10 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent bootstrapping tablet: real 0.178s user 0.022s sys 0.007s
I20260808 13:36:03.620649 6033 raft_consensus.cc:348] T d439b27c3ad34619972c71d652f1ff10 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
I20260808 13:36:03.625393 6033 raft_consensus.cc:374] T d439b27c3ad34619972c71d652f1ff10 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:03.626968 6033 raft_consensus.cc:729] T d439b27c3ad34619972c71d652f1ff10 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dd96f425fc4ab3a6a54f2dfaaf9fab, State: Initialized, Role: FOLLOWER
I20260808 13:36:03.630072 6033 consensus_queue.cc:260] T d439b27c3ad34619972c71d652f1ff10 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98dd96f425fc4ab3a6a54f2dfaaf9fab" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 34173 } } peers { permanent_uuid: "5cd987c3dbdb4168beb73cd7e577c7cb" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 40577 } } peers { permanent_uuid: "946c9d8462644deba5b45fa724f1923c" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 40797 } }
W20260808 13:36:03.637166 5771 consensus_peers.cc:487] T b0f778e7fa51414cb209fbe1aea92739 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.1:40797: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260808 13:36:03.637678 5771 consensus_peers.cc:487] T 9f53e5b5c4bd4a9385ad198b283c6e79 P 98dd96f425fc4ab3a6a54f2dfaaf9fab -> Peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Couldn't send request to peer 946c9d8462644deba5b45fa724f1923c. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.1:40797: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260808 13:36:03.638031 5771 leader_election.cc:334] T 604d2113451642329b877f8beff71618 P 98dd96f425fc4ab3a6a54f2dfaaf9fab [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 946c9d8462644deba5b45fa724f1923c (127.4.170.1:40797): Network error: Client connection negotiation failed: client connection to 127.4.170.1:40797: connect: Connection refused (error 111)
I20260808 13:36:03.641419 6033 ts_tablet_manager.cc:1428] T d439b27c3ad34619972c71d652f1ff10 P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Time spent starting tablet: real 0.040s user 0.012s sys 0.000s
I20260808 13:36:03.653263 6033 tablet_bootstrap.cc:492] T 293e9540498e420d8ec873472d0d29fc P 98dd96f425fc4ab3a6a54f2dfaaf9fab: Bootstrap starting.
W20260808 13:36:03.655262 5772 connection.cc:535] client connection to 127.4.170.2:40577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 13:36:03.658288 4776 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu with pid 5737
W20260808 13:36:03.848829 4821 connection.cc:535] server connection from 127.4.170.3:51951 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 13:36:03.850521 4776 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu with pid 4791
I20260808 13:36:03.855677 4776 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu with pid 4904
W20260808 13:36:03.945218 7535 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.170.61:46555 (error 108) [suppressed 2 similar messages]
I20260808 13:36:03.977633 4776 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu with pid 6450
2026-08-08T13:36:04Z chronyd exiting
I20260808 13:36:04.070478 4776 master-stress-test.cc:586] Tables created: 252
I20260808 13:36:04.070744 4776 master-stress-test.cc:587] Tables altered: 111
I20260808 13:36:04.070878 4776 master-stress-test.cc:588] Tables deleted: 98
I20260808 13:36:04.071065 4776 master-stress-test.cc:589] Tablets replaced: 26
I20260808 13:36:04.071201 4776 master-stress-test.cc:590] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (83299 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
I20260808 13:36:04.209214 4776 test_util.cc:268] Using random seed: 870181412
2026-08-08T13:36:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:36:04Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskoVZaNS/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskoVZaNS/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/security.properties
2026-08-08 13:36:11: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskoVZaNS/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskoVZaNS/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260808 13:36:18.931182 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.62:46707
--webserver_interface=127.4.170.62
--webserver_port=0
--builtin_ntp_servers=127.4.170.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--hive_metastore_uris=thrift://127.0.0.1:46603
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260808 13:36:19.263502 9774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:36:19.264163 9774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:36:19.264671 9774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:36:19.297663 9774 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260808 13:36:19.297998 9774 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260808 13:36:19.298275 9774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:36:19.298534 9774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:36:19.298856 9774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:36:19.299106 9774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:36:19.335172 9774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:46603
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.62:46707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.170.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:36:19.336483 9774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:36:19.338140 9774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:36:19.357120 9793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:36:19.357229 9791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:19.358840 9774 server_base.cc:1034] running on GCE node
W20260808 13:36:19.357335 9787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:20.554288 9774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:36:20.557744 9774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:36:20.559275 9774 hybrid_clock.cc:648] HybridClock initialized: now 1786196180559238 us; error 56 us; skew 500 ppm
I20260808 13:36:20.560199 9774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:36:20.576550 9774 webserver.cc:460] Webserver started at http://127.4.170.62:33447/ using document root <none> and password file <none>
I20260808 13:36:20.577597 9774 fs_manager.cc:359] Metadata directory not provided
I20260808 13:36:20.577844 9774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:36:20.578382 9774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:36:20.583577 9774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/data/instance:
uuid: "bc2c297fb28f4950864f0487ef83e833"
format_stamp: "Formatted at 2026-08-08 13:36:20 on dist-test-slave-twwt"
I20260808 13:36:20.584743 9774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/wal/instance:
uuid: "bc2c297fb28f4950864f0487ef83e833"
format_stamp: "Formatted at 2026-08-08 13:36:20 on dist-test-slave-twwt"
I20260808 13:36:20.592347 9774 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260808 13:36:20.598724 9804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:20.599861 9774 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260808 13:36:20.600217 9774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/wal
uuid: "bc2c297fb28f4950864f0487ef83e833"
format_stamp: "Formatted at 2026-08-08 13:36:20 on dist-test-slave-twwt"
I20260808 13:36:20.600598 9774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:36:20.636096 9774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:36:20.637699 9774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:36:20.638226 9774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:36:20.715699 9774 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.62:46707
I20260808 13:36:20.715826 9892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.62:46707 every 8 connection(s)
I20260808 13:36:20.718515 9774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/data/info.pb
I20260808 13:36:20.724494 9893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:20.725683 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 9774
I20260808 13:36:20.726243 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-0/wal/instance
I20260808 13:36:20.730090 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.61:37949
--webserver_interface=127.4.170.61
--webserver_port=0
--builtin_ntp_servers=127.4.170.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--hive_metastore_uris=thrift://127.0.0.1:46603
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260808 13:36:20.743736 9893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:36:20.776274 9893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } has no permanent_uuid. Determining permanent_uuid...
W20260808 13:36:20.778939 9805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
W20260808 13:36:20.783510 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:37949: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:20.833055 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } attempt: 1
W20260808 13:36:20.837440 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:37949: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:20.905861 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } attempt: 2
W20260808 13:36:20.909775 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:37949: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:21.001230 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } attempt: 3
W20260808 13:36:21.005311 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:37949: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
W20260808 13:36:21.073920 9894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:36:21.074565 9894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:36:21.075081 9894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:36:21.106884 9894 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260808 13:36:21.107164 9894 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260808 13:36:21.107450 9894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:36:21.107707 9894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:36:21.107967 9894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:36:21.108214 9894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:36:21.145093 9894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:46603
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.61:37949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.170.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:36:21.146431 9894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:36:21.148144 9894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 13:36:21.148821 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } attempt: 4
W20260808 13:36:21.152614 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:37949: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
W20260808 13:36:21.167273 9912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:21.452090 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } attempt: 5
W20260808 13:36:21.456650 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:37949: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:22.004065 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } attempt: 6
W20260808 13:36:22.012293 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.61:37949: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
W20260808 13:36:21.167708 9914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:36:21.167732 9916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:36:22.372983 9915 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260808 13:36:22.373092 9894 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260808 13:36:22.376962 9894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:36:22.379660 9894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:36:22.381031 9894 hybrid_clock.cc:648] HybridClock initialized: now 1786196182381005 us; error 58 us; skew 500 ppm
I20260808 13:36:22.382169 9894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:36:22.388741 9894 webserver.cc:460] Webserver started at http://127.4.170.61:42377/ using document root <none> and password file <none>
I20260808 13:36:22.389714 9894 fs_manager.cc:359] Metadata directory not provided
I20260808 13:36:22.389921 9894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:36:22.390362 9894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:36:22.396096 9894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data/instance:
uuid: "026dcc55447e4441b062e327f64eacb4"
format_stamp: "Formatted at 2026-08-08 13:36:22 on dist-test-slave-twwt"
I20260808 13:36:22.397239 9894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal/instance:
uuid: "026dcc55447e4441b062e327f64eacb4"
format_stamp: "Formatted at 2026-08-08 13:36:22 on dist-test-slave-twwt"
I20260808 13:36:22.404839 9894 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260808 13:36:22.410544 9929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:22.411705 9894 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 13:36:22.412055 9894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal
uuid: "026dcc55447e4441b062e327f64eacb4"
format_stamp: "Formatted at 2026-08-08 13:36:22 on dist-test-slave-twwt"
I20260808 13:36:22.412395 9894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:36:22.475508 9894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:36:22.477059 9894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:36:22.477560 9894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:36:22.551823 9894 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.61:37949
I20260808 13:36:22.551914 10022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.61:37949 every 8 connection(s)
I20260808 13:36:22.554716 9894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data/info.pb
I20260808 13:36:22.559753 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 9894
I20260808 13:36:22.560278 10024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:22.560497 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal/instance
I20260808 13:36:22.564602 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.60:45415
--webserver_interface=127.4.170.60
--webserver_port=0
--builtin_ntp_servers=127.4.170.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--hive_metastore_uris=thrift://127.0.0.1:46603
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260808 13:36:22.579442 10024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:36:22.604036 10024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:36:22.621414 10024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } has no permanent_uuid. Determining permanent_uuid...
W20260808 13:36:22.624467 9939 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:36:22.628783 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:22.678383 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 1
W20260808 13:36:22.682415 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:22.750999 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 2
W20260808 13:36:22.754997 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:22.846498 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 3
W20260808 13:36:22.850808 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:36:22.916971 10026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:36:22.917536 10026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:36:22.917950 10026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:36:22.949630 10026 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260808 13:36:22.949976 10026 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260808 13:36:22.950237 10026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:36:22.950435 10026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:36:22.950644 10026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:36:22.950832 10026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:36:22.992220 10026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:46603
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.60:45415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.170.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:36:22.993947 10026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:36:22.994282 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 4
I20260808 13:36:22.996055 10026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:36:22.999293 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:36:23.018882 10045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:23.072672 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } attempt: 7
I20260808 13:36:23.089018 9893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } has no permanent_uuid. Determining permanent_uuid...
W20260808 13:36:23.094519 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:23.153234 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 1
W20260808 13:36:23.157603 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:23.239220 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 2
W20260808 13:36:23.243425 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:23.298779 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 5
W20260808 13:36:23.303715 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:23.329070 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 3
W20260808 13:36:23.334362 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:23.474860 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 4
W20260808 13:36:23.479501 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:23.762941 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 5
W20260808 13:36:23.767755 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:23.851127 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 6
W20260808 13:36:23.859138 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:36:23.018888 10042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:36:23.021710 10040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:36:24.248569 10043 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260808 13:36:24.248659 10026 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260808 13:36:24.252508 10026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:36:24.255875 10026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:36:24.257319 10026 hybrid_clock.cc:648] HybridClock initialized: now 1786196184257252 us; error 88 us; skew 500 ppm
I20260808 13:36:24.258209 10026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:36:24.265398 10026 webserver.cc:460] Webserver started at http://127.4.170.60:40025/ using document root <none> and password file <none>
I20260808 13:36:24.266358 10026 fs_manager.cc:359] Metadata directory not provided
I20260808 13:36:24.266593 10026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:36:24.267141 10026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:36:24.272444 10026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data/instance:
uuid: "0bf9978620f543d8aee6279b9a9d3534"
format_stamp: "Formatted at 2026-08-08 13:36:24 on dist-test-slave-twwt"
I20260808 13:36:24.273635 10026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal/instance:
uuid: "0bf9978620f543d8aee6279b9a9d3534"
format_stamp: "Formatted at 2026-08-08 13:36:24 on dist-test-slave-twwt"
I20260808 13:36:24.281421 10026 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260808 13:36:24.287437 10067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:24.288475 10026 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260808 13:36:24.288893 10026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal
uuid: "0bf9978620f543d8aee6279b9a9d3534"
format_stamp: "Formatted at 2026-08-08 13:36:24 on dist-test-slave-twwt"
I20260808 13:36:24.289295 10026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:36:24.320344 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 6
W20260808 13:36:24.327060 9893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.170.60:45415: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:24.338142 10026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:36:24.339740 10026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:36:24.340301 10026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:36:24.415439 10026 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.60:45415
I20260808 13:36:24.415531 10153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.60:45415 every 8 connection(s)
I20260808 13:36:24.418130 10026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
I20260808 13:36:24.422068 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 10026
I20260808 13:36:24.422523 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal/instance
I20260808 13:36:24.423636 10154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:24.442214 10154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:36:24.468047 10154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:36:24.480052 10154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:36:24.499989 10154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Bootstrap starting.
I20260808 13:36:24.505844 10154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:24.507649 10154 log.cc:824] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Log is configured to *not* fsync() on all Append() calls
I20260808 13:36:24.512763 10154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: No bootstrap required, opened a new log
I20260808 13:36:24.533990 10154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:24.534821 10154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:24.535095 10154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf9978620f543d8aee6279b9a9d3534, State: Initialized, Role: FOLLOWER
I20260808 13:36:24.535862 10154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:24.538277 10165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:24.538998 10165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:24.539728 10154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:36:24.584314 10165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:24.584890 10165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
W20260808 13:36:24.587387 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.587776 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:24.590516 10165 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949)
W20260808 13:36:24.595639 10080 tablet.cc:2369] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:36:24.602207 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.602566 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:24.574034 10026 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: OK, instance_id { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" instance_seqno: 1786196180693587 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:36:24.620461 10026 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:36:24.621230 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.621886 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.645327 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.646392 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.660944 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.661281 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.675558 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.675880 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.703325 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.704393 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.719313 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.719695 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.734189 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.734740 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.750392 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.750725 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.765290 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.765666 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.780094 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.780433 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.794831 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.795178 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.809499 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.809823 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.824003 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.824338 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.838598 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.838939 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.853053 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.853447 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.867973 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.868310 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.883700 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.884109 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.898638 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.898990 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.913828 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.914134 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:24.919513 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 7
W20260808 13:36:24.929591 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.930017 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:24.946733 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.947814 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:24.958388 10024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Bootstrap starting.
W20260808 13:36:24.962188 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.962491 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:24.965675 10024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:24.968127 10024 log.cc:824] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Log is configured to *not* fsync() on all Append() calls
W20260808 13:36:24.976572 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.976859 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:24.980609 10024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: No bootstrap required, opened a new log
W20260808 13:36:24.991405 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:24.991742 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:25.006330 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:25.006687 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.013231 10024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:25.014168 10024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:25.014534 10024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 026dcc55447e4441b062e327f64eacb4, State: Initialized, Role: FOLLOWER
I20260808 13:36:25.015664 10024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:25.018464 10196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:25.019361 10196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:25.020754 10024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:36:25.021071 9985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4" is_pre_election: true
W20260808 13:36:25.022898 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:25.023277 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.022856 9985 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf9978620f543d8aee6279b9a9d3534 in term 0.
I20260808 13:36:25.024797 10068 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534; no voters:
I20260808 13:36:25.025671 10165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:25.025987 10165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:25.026283 10165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:25.034289 10165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
W20260808 13:36:25.041695 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:25.041989 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.045526 9985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4"
I20260808 13:36:25.046130 9985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:25.055954 9985 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf9978620f543d8aee6279b9a9d3534 in term 1.
I20260808 13:36:25.057067 10068 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534; no voters:
I20260808 13:36:25.059789 10165 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 1 election: Requested vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949)
I20260808 13:36:25.060847 10211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:25.062244 10211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 1 LEADER]: Becoming Leader. State: Replica: 0bf9978620f543d8aee6279b9a9d3534, State: Running, Role: LEADER
I20260808 13:36:25.063390 10211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
W20260808 13:36:25.072861 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:25.073151 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.075799 10165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf9978620f543d8aee6279b9a9d3534. Latest consensus state: current_term: 1 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:25.076495 10165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: LEADER
W20260808 13:36:25.086491 10186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:25.086899 10186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.087265 10215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:36:25.104308 10215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:36:25.136385 9985 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Refusing update from remote peer 0bf9978620f543d8aee6279b9a9d3534: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:36:25.140769 10165 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [LEADER]: Connected to new peer: Peer: permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:36:25.141018 10222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:25.143006 10222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.111668 9894 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: OK, instance_id { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" instance_seqno: 1786196180693587 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:36:25.184942 9894 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:36:25.202443 10225 mvcc.cc:205] Tried to move back new op lower bound from 7316259574273998848 to 7316259574030241792. Current Snapshot: MvccSnapshot[applied={T|T < 7316259574273998848}]
I20260808 13:36:25.203676 10196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf9978620f543d8aee6279b9a9d3534. Latest consensus state: current_term: 1 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:25.204818 10196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 13:36:25.213778 10222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:36:25.215797 10222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.222191 10211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:25.223016 10211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:25.230928 10215 catalog_manager.cc:1347] Generated new cluster ID: 9eb2bfe6721b4b6487ca35bc443651ed
I20260808 13:36:25.231240 10215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260808 13:36:25.236786 10073 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.170.62:46707 timed out after 0.647s (SENT)
I20260808 13:36:25.247309 10196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:25.248206 10196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:25.251817 10222 catalog_manager.cc:1259] Loaded cluster ID: 9eb2bfe6721b4b6487ca35bc443651ed
I20260808 13:36:25.252168 10222 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: loading cluster ID for follower catalog manager: success
W20260808 13:36:25.256778 10222 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 13:36:25.257190 10222 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:25.272346 10222 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 13:36:25.272810 10222 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:25.287971 10222 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 13:36:25.288368 10222 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:25.303459 10222 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 13:36:25.303884 10222 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:25.327716 10222 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 13:36:25.328128 10222 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260808 13:36:25.343542 10222 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 13:36:25.343957 10222 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.360488 9893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } attempt: 7
I20260808 13:36:25.372975 10215 catalog_manager.cc:1370] Generated new certificate authority record
W20260808 13:36:25.378720 10222 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 13:36:25.379146 10222 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.393738 10215 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:36:25.397951 10222 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: acquiring CA information for follower catalog manager: success
W20260808 13:36:25.407383 10222 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.423816 9893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: Bootstrap starting.
W20260808 13:36:25.426532 10222 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.442422 9893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: Neither blocks nor log segments found. Creating new log.
W20260808 13:36:25.443470 10222 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.445070 9893 log.cc:824] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: Log is configured to *not* fsync() on all Append() calls
W20260808 13:36:25.458737 10222 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260808 13:36:25.464485 9893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: No bootstrap required, opened a new log
I20260808 13:36:25.471904 10215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Generated new TSK 0
I20260808 13:36:25.476816 10222 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:36:25.477228 10215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:36:25.495935 10215 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260808 13:36:25.523874 10215 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260808 13:36:25.525760 9893 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:25.526710 9893 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:25.527070 9893 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc2c297fb28f4950864f0487ef83e833, State: Initialized, Role: FOLLOWER
I20260808 13:36:25.528122 9893 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:25.539742 10238 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:25.542029 9893 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:36:25.550777 10238 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:25.568883 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.170.1:0
--local_ip_for_outbound_sockets=127.4.170.1
--webserver_interface=127.4.170.1
--webserver_port=0
--tserver_master_addrs=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--builtin_ntp_servers=127.4.170.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260808 13:36:25.542960 9850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:25.543524 9848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833" is_pre_election: true
I20260808 13:36:25.544258 9851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833"
W20260808 13:36:25.576246 9851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46896 (request call id 3) took 539 ms (client timeout 715 ms). Trace:
W20260808 13:36:25.576608 9851 rpcz_store.cc:269] 0808 13:36:25.036802 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:36:25.036979 (+ 177us) service_pool.cc:227] Handling call
0808 13:36:25.576170 (+539191us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:36:25.544857 9855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833" is_pre_election: true
W20260808 13:36:25.578110 9855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46896 (request call id 2) took 988 ms (client timeout 647 ms). Trace:
W20260808 13:36:25.578413 9855 rpcz_store.cc:269] 0808 13:36:24.589762 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:36:24.589909 (+ 147us) service_pool.cc:227] Handling call
0808 13:36:25.578052 (+988143us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260808 13:36:25.609030 9850 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 1 FOLLOWER]: Refusing update from remote peer 0bf9978620f543d8aee6279b9a9d3534: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:36:25.611116 10211 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:36:25.654815 9813 tablet.cc:2369] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:36:25.706297 10250 mvcc.cc:205] Tried to move back new op lower bound from 7316259574273998848 to 7316259574030241792. Current Snapshot: MvccSnapshot[applied={T|T < 7316259574273998848}]
I20260808 13:36:25.736246 10238 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf9978620f543d8aee6279b9a9d3534. Latest consensus state: current_term: 1 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:25.737048 10238 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:25.739475 10254 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:25.740418 10254 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:25.779642 10270 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 13:36:25.830426 10270 catalog_manager.cc:1259] Loaded cluster ID: 9eb2bfe6721b4b6487ca35bc443651ed
I20260808 13:36:25.832227 10270 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: loading cluster ID for follower catalog manager: success
I20260808 13:36:25.848922 10270 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: acquiring CA information for follower catalog manager: success
I20260808 13:36:25.857635 10270 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:36:26.090098 10211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:26.090898 10211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: LEADER
W20260808 13:36:26.417366 10249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:36:26.418036 10249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:36:26.419008 10249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:36:26.488780 10249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:36:26.490324 10249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.170.1
I20260808 13:36:26.586916 10249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.170.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.170.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.170.1
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:36:26.589839 10249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:36:26.592047 10249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:36:26.659574 10285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:27.299482 10292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:27.300076 10292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:27.303412 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833" is_pre_election: true
I20260808 13:36:27.306860 10116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:27.308712 9939 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:27.314289 10292 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:27.319169 10292 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:36:27.858803 10298 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:27.860183 10298 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:27.861439 10297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:27.866201 10297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:27.869344 9982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4" is_pre_election: true
I20260808 13:36:27.869941 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833" is_pre_election: true
I20260808 13:36:27.870645 9850 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 026dcc55447e4441b062e327f64eacb4 in term 1.
I20260808 13:36:27.870762 9982 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc2c297fb28f4950864f0487ef83e833 in term 1.
I20260808 13:36:27.873277 9940 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4, bc2c297fb28f4950864f0487ef83e833; no voters:
I20260808 13:36:27.874519 9805 leader_election.cc:302] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4, bc2c297fb28f4950864f0487ef83e833; no voters:
I20260808 13:36:27.877147 10298 leader_election.cc:288] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:27.877837 10298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:36:27.878212 10298 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:27.878257 10116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:27.878659 10298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:36:27.880658 10299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:36:27.883411 10116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:27.885434 10299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:27.885886 10299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:36:27.883798 10297 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:27.898783 10298 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:27.902443 10298 leader_election.cc:288] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 2 election: Requested vote from peers 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:27.902997 9982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4"
I20260808 13:36:27.913717 10116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534"
I20260808 13:36:27.915928 10299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:27.918360 9982 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bc2c297fb28f4950864f0487ef83e833 in current term 2: Already voted for candidate 026dcc55447e4441b062e327f64eacb4 in this term.
I20260808 13:36:27.919200 10116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534"
I20260808 13:36:27.920809 9985 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 2 FOLLOWER]: Rejecting Update request from peer 0bf9978620f543d8aee6279b9a9d3534 for earlier term 1. Current term is 2. Ops: []
I20260808 13:36:27.922174 9805 leader_election.cc:302] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc2c297fb28f4950864f0487ef83e833; no voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534
I20260808 13:36:27.923276 10298 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 13:36:27.923514 10301 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260808 13:36:27.924065 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833"
I20260808 13:36:27.924505 10299 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 2 election: Requested vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:27.924849 9850 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 026dcc55447e4441b062e327f64eacb4 in current term 2: Already voted for candidate bc2c297fb28f4950864f0487ef83e833 in this term.
I20260808 13:36:27.926081 9940 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:27.930787 10299 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 13:36:27.934475 10303 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 1 LEADER]: Stepping down as leader of term 1
I20260808 13:36:27.937844 10303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bf9978620f543d8aee6279b9a9d3534, State: Running, Role: LEADER
I20260808 13:36:27.939865 10303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:27.943068 10303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:36:27.967696 10186 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 13:36:26.661285 10288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:26.667454 10249 server_base.cc:1034] running on GCE node
W20260808 13:36:26.660471 10283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:28.224208 10249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:36:28.227375 10249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:36:28.229003 10249 hybrid_clock.cc:648] HybridClock initialized: now 1786196188228927 us; error 101 us; skew 500 ppm
I20260808 13:36:28.230167 10249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:36:28.242341 10249 webserver.cc:460] Webserver started at http://127.4.170.1:34773/ using document root <none> and password file <none>
I20260808 13:36:28.243703 10249 fs_manager.cc:359] Metadata directory not provided
I20260808 13:36:28.243999 10249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:36:28.244596 10249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:36:28.252239 10249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/data/instance:
uuid: "117d8c07fa254197969de761dd678aa6"
format_stamp: "Formatted at 2026-08-08 13:36:28 on dist-test-slave-twwt"
I20260808 13:36:28.253854 10249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/instance:
uuid: "117d8c07fa254197969de761dd678aa6"
format_stamp: "Formatted at 2026-08-08 13:36:28 on dist-test-slave-twwt"
I20260808 13:36:28.263713 10249 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.004s
I20260808 13:36:28.271477 10308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:28.272878 10249 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260808 13:36:28.273264 10249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal
uuid: "117d8c07fa254197969de761dd678aa6"
format_stamp: "Formatted at 2026-08-08 13:36:28 on dist-test-slave-twwt"
I20260808 13:36:28.273705 10249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:36:28.325785 10249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:36:28.327853 10249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:36:28.328419 10249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:36:28.332127 10249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:36:28.338310 10249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:36:28.338589 10249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:28.338953 10249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:36:28.339169 10249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:28.343478 10303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:28.343881 10303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:28.345603 10303 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949)
I20260808 13:36:28.346832 9985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4" is_pre_election: true
I20260808 13:36:28.346848 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833" is_pre_election: true
I20260808 13:36:28.347358 9985 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf9978620f543d8aee6279b9a9d3534 in term 2.
I20260808 13:36:28.347447 9850 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf9978620f543d8aee6279b9a9d3534 in term 2.
I20260808 13:36:28.348621 10068 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534; no voters:
I20260808 13:36:28.349406 10303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 13:36:28.349721 10303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:28.350041 10303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:36:28.356308 10303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:28.359489 9985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4"
I20260808 13:36:28.360047 9985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:36:28.360603 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833"
I20260808 13:36:28.361171 9850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:36:28.366835 9985 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf9978620f543d8aee6279b9a9d3534 in term 3.
I20260808 13:36:28.367673 9850 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf9978620f543d8aee6279b9a9d3534 in term 3.
I20260808 13:36:28.369058 10068 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534; no voters:
I20260808 13:36:28.369907 10301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 13:36:28.370015 10303 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 3 election: Requested vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949)
I20260808 13:36:28.371358 10301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 3 LEADER]: Becoming Leader. State: Replica: 0bf9978620f543d8aee6279b9a9d3534, State: Running, Role: LEADER
I20260808 13:36:28.372288 10301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:28.378645 10303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf9978620f543d8aee6279b9a9d3534. Latest consensus state: current_term: 3 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:28.379613 10303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:28.382387 10329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:36:28.391188 10329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:36:28.395504 10329 catalog_manager.cc:1259] Loaded cluster ID: 9eb2bfe6721b4b6487ca35bc443651ed
I20260808 13:36:28.395857 10329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:36:28.401921 10329 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:36:28.406183 10329 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Loaded TSK: 0
I20260808 13:36:28.407755 10329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:36:28.412360 10329 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260808 13:36:28.416266 10329 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260808 13:36:28.574309 10249 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.1:35471
I20260808 13:36:28.574455 10490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.1:35471 every 8 connection(s)
I20260808 13:36:28.578136 10249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/data/info.pb
I20260808 13:36:28.579779 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 10249
I20260808 13:36:28.580322 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/instance
I20260808 13:36:28.595295 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.170.2:0
--local_ip_for_outbound_sockets=127.4.170.2
--webserver_interface=127.4.170.2
--webserver_port=0
--tserver_master_addrs=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--builtin_ntp_servers=127.4.170.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260808 13:36:28.678360 10496 heartbeater.cc:347] Connected to a master server at 127.4.170.62:46707
I20260808 13:36:28.678936 10492 heartbeater.cc:347] Connected to a master server at 127.4.170.60:45415
I20260808 13:36:28.679332 10492 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:28.678938 10496 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:28.680560 10492 heartbeater.cc:511] Master 127.4.170.60:45415 requested a full tablet report, sending...
I20260808 13:36:28.680760 10496 heartbeater.cc:511] Master 127.4.170.62:46707 requested a full tablet report, sending...
I20260808 13:36:28.684259 9832 ts_manager.cc:194] Registered new tserver with Master: 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471)
I20260808 13:36:28.684231 10097 ts_manager.cc:194] Registered new tserver with Master: 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471)
I20260808 13:36:28.685801 10494 heartbeater.cc:347] Connected to a master server at 127.4.170.61:37949
I20260808 13:36:28.686175 10494 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:28.687233 10494 heartbeater.cc:511] Master 127.4.170.61:37949 requested a full tablet report, sending...
I20260808 13:36:28.689237 10097 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.170.1:57413
I20260808 13:36:28.690766 9967 ts_manager.cc:194] Registered new tserver with Master: 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471)
I20260808 13:36:28.792488 9850 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Refusing update from remote peer 0bf9978620f543d8aee6279b9a9d3534: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260808 13:36:28.794143 10303 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260808 13:36:28.801069 10510 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf9978620f543d8aee6279b9a9d3534. Latest consensus state: current_term: 3 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:28.801973 10510 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:28.818365 10510 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:28.819201 10510 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:28.823405 10301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:28.824256 10301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:28.824977 9985 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Refusing update from remote peer 0bf9978620f543d8aee6279b9a9d3534: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260808 13:36:28.826568 10301 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [LEADER]: Connected to new peer: Peer: permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260808 13:36:28.842460 10521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:28.842909 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf9978620f543d8aee6279b9a9d3534. Latest consensus state: current_term: 3 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:28.843279 10521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:28.843600 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 13:36:29.042922 10507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:36:29.043484 10507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:36:29.044118 10507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:36:29.078684 10507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:36:29.080046 10507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.170.2
I20260808 13:36:29.117889 10507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.170.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.170.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.170.2
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:36:29.119611 10507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:36:29.121438 10507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:36:29.142831 10536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:29.193945 10492 heartbeater.cc:503] Master 127.4.170.60:45415 was elected leader, sending a full tablet report...
I20260808 13:36:29.224128 10303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:29.225274 10303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:29.727274 10547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:29.728341 10547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:29.731148 10547 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:29.732286 10116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:29.734824 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833" is_pre_election: true
I20260808 13:36:29.737710 9940 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:29.739281 10547 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:36:29.774020 10549 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:29.774818 10549 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:29.776988 10549 leader_election.cc:288] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:29.777693 10116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:29.778023 9985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4" is_pre_election: true
I20260808 13:36:29.780040 9805 leader_election.cc:302] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc2c297fb28f4950864f0487ef83e833; no voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534
I20260808 13:36:29.780908 10549 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:36:30.275228 10550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:30.275774 10550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:30.277377 10550 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:30.278385 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833" is_pre_election: true
I20260808 13:36:30.278457 10116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:30.279819 9940 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:30.280439 10550 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260808 13:36:29.143505 10531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:29.145125 10507 server_base.cc:1034] running on GCE node
W20260808 13:36:29.142992 10532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:30.385677 10507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:36:30.388083 10507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:36:30.389456 10507 hybrid_clock.cc:648] HybridClock initialized: now 1786196190389400 us; error 45 us; skew 500 ppm
I20260808 13:36:30.390311 10507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:36:30.397581 10507 webserver.cc:460] Webserver started at http://127.4.170.2:39443/ using document root <none> and password file <none>
I20260808 13:36:30.398548 10507 fs_manager.cc:359] Metadata directory not provided
I20260808 13:36:30.398835 10507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:36:30.399338 10507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:36:30.404726 10507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/data/instance:
uuid: "e9d3df921f204d8aa02da52aca33d995"
format_stamp: "Formatted at 2026-08-08 13:36:30 on dist-test-slave-twwt"
I20260808 13:36:30.405889 10507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/instance:
uuid: "e9d3df921f204d8aa02da52aca33d995"
format_stamp: "Formatted at 2026-08-08 13:36:30 on dist-test-slave-twwt"
I20260808 13:36:30.413916 10507 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.001s
I20260808 13:36:30.420661 10558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:30.422109 10507 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:36:30.422487 10507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal
uuid: "e9d3df921f204d8aa02da52aca33d995"
format_stamp: "Formatted at 2026-08-08 13:36:30 on dist-test-slave-twwt"
I20260808 13:36:30.423035 10507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:36:30.485407 10507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:36:30.487105 10507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:36:30.487583 10507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:36:30.490295 10507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:36:30.494840 10507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:36:30.495079 10507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:30.495335 10507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:36:30.495515 10507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:30.657821 10507 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.2:34841
I20260808 13:36:30.657979 10729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.2:34841 every 8 connection(s)
I20260808 13:36:30.660449 10507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/data/info.pb
I20260808 13:36:30.664634 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 10507
I20260808 13:36:30.665230 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/instance
I20260808 13:36:30.686878 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.170.3:0
--local_ip_for_outbound_sockets=127.4.170.3
--webserver_interface=127.4.170.3
--webserver_port=0
--tserver_master_addrs=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--builtin_ntp_servers=127.4.170.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260808 13:36:30.720642 10730 heartbeater.cc:347] Connected to a master server at 127.4.170.60:45415
I20260808 13:36:30.721187 10730 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:30.722669 10730 heartbeater.cc:511] Master 127.4.170.60:45415 requested a full tablet report, sending...
I20260808 13:36:30.725824 10097 ts_manager.cc:194] Registered new tserver with Master: e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:30.728742 10731 heartbeater.cc:347] Connected to a master server at 127.4.170.61:37949
I20260808 13:36:30.729122 10731 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:30.729990 10731 heartbeater.cc:511] Master 127.4.170.61:37949 requested a full tablet report, sending...
I20260808 13:36:30.732757 9967 ts_manager.cc:194] Registered new tserver with Master: e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:30.739009 10097 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.170.2:35879
I20260808 13:36:30.741092 10732 heartbeater.cc:347] Connected to a master server at 127.4.170.62:46707
I20260808 13:36:30.741464 10732 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:30.742302 10732 heartbeater.cc:511] Master 127.4.170.62:46707 requested a full tablet report, sending...
I20260808 13:36:30.745008 9832 ts_manager.cc:194] Registered new tserver with Master: e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:36:31.066927 10741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:36:31.067472 10741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:36:31.068140 10741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:36:31.101562 10741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:36:31.102550 10741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.170.3
I20260808 13:36:31.139433 10741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.170.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.170.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.170.3
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:36:31.140842 10741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:36:31.142436 10741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:36:31.163515 10757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:31.245040 10730 heartbeater.cc:503] Master 127.4.170.60:45415 was elected leader, sending a full tablet report...
I20260808 13:36:32.332145 10768 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:32.332666 10768 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:32.334913 10768 leader_election.cc:288] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:32.335662 9985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4" is_pre_election: true
I20260808 13:36:32.335652 10116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:32.337342 9809 leader_election.cc:302] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc2c297fb28f4950864f0487ef83e833; no voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534
I20260808 13:36:32.337991 10768 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260808 13:36:31.163770 10756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:36:31.164362 10762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:31.166095 10741 server_base.cc:1034] running on GCE node
I20260808 13:36:32.414179 10741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:36:32.417268 10741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:36:32.418741 10741 hybrid_clock.cc:648] HybridClock initialized: now 1786196192418698 us; error 59 us; skew 500 ppm
I20260808 13:36:32.419612 10741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:36:32.427778 10741 webserver.cc:460] Webserver started at http://127.4.170.3:42469/ using document root <none> and password file <none>
I20260808 13:36:32.428864 10741 fs_manager.cc:359] Metadata directory not provided
I20260808 13:36:32.429090 10741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:36:32.429607 10741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:36:32.435524 10741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/data/instance:
uuid: "e0e40695b6c9401aa0c178f58d8e2f37"
format_stamp: "Formatted at 2026-08-08 13:36:32 on dist-test-slave-twwt"
I20260808 13:36:32.436803 10741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/instance:
uuid: "e0e40695b6c9401aa0c178f58d8e2f37"
format_stamp: "Formatted at 2026-08-08 13:36:32 on dist-test-slave-twwt"
I20260808 13:36:32.444509 10741 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260808 13:36:32.450938 10788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:32.452294 10741 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260808 13:36:32.452780 10741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal
uuid: "e0e40695b6c9401aa0c178f58d8e2f37"
format_stamp: "Formatted at 2026-08-08 13:36:32 on dist-test-slave-twwt"
I20260808 13:36:32.453188 10741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:36:32.492307 10741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:36:32.493907 10741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:36:32.494416 10741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:36:32.497154 10741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:36:32.501402 10741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:36:32.501592 10741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:32.501899 10741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:36:32.502067 10741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:32.671250 10741 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.3:35889
I20260808 13:36:32.671375 10986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.3:35889 every 8 connection(s)
I20260808 13:36:32.674146 10741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/data/info.pb
I20260808 13:36:32.682958 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 10741
I20260808 13:36:32.683987 4776 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/instance
I20260808 13:36:32.755896 10989 heartbeater.cc:347] Connected to a master server at 127.4.170.61:37949
I20260808 13:36:32.756441 10989 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:32.757738 10989 heartbeater.cc:511] Master 127.4.170.61:37949 requested a full tablet report, sending...
I20260808 13:36:32.762488 10987 heartbeater.cc:347] Connected to a master server at 127.4.170.60:45415
I20260808 13:36:32.762398 9967 ts_manager.cc:194] Registered new tserver with Master: e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:32.762880 10987 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:32.763777 10987 heartbeater.cc:511] Master 127.4.170.60:45415 requested a full tablet report, sending...
I20260808 13:36:32.765956 10096 ts_manager.cc:194] Registered new tserver with Master: e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:32.766435 10991 heartbeater.cc:347] Connected to a master server at 127.4.170.62:46707
I20260808 13:36:32.766794 10991 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:32.767628 10991 heartbeater.cc:511] Master 127.4.170.62:46707 requested a full tablet report, sending...
I20260808 13:36:32.768630 10096 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.170.3:36443
I20260808 13:36:32.770262 9832 ts_manager.cc:194] Registered new tserver with Master: e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:32.779898 4776 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:36:32.833801 10096 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39920:
name: "default.Table_e5e780904d7a4ad0a2b5ba7e99c1de0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:32.834079 10097 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39920:
name: "default.Table_82a5f186cbcd4cf0875234550bb2665e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:32.838526 10094 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39920:
name: "default.Table_f7b46abc59bd4cacacf793d1d5a184b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:32.841187 10092 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39920:
name: "default.Table_575a74e193bc45fd99dcd06ae9318532"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:36:32.850773 10096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_e5e780904d7a4ad0a2b5ba7e99c1de0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:36:32.867446 10097 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_82a5f186cbcd4cf0875234550bb2665e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:36:32.867885 10092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_575a74e193bc45fd99dcd06ae9318532 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:36:32.869355 10094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f7b46abc59bd4cacacf793d1d5a184b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:36:32.876964 4776 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu with pid 10026
W20260808 13:36:32.941537 11009 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.170.60:45415)
I20260808 13:36:32.963515 11049 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:32.963958 11049 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:32.966063 11049 leader_election.cc:288] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:32.966445 9985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4" is_pre_election: true
W20260808 13:36:32.981971 9809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111) [suppressed 13 similar messages]
W20260808 13:36:32.988492 9809 leader_election.cc:334] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:32.988998 9809 leader_election.cc:302] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc2c297fb28f4950864f0487ef83e833; no voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534
I20260808 13:36:32.989751 11049 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260808 13:36:33.135720 11054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:33.136439 11054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:33.141866 11054 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:33.142180 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833" is_pre_election: true
I20260808 13:36:33.142884 9850 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 026dcc55447e4441b062e327f64eacb4 in term 3.
W20260808 13:36:33.144804 9939 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260808 13:36:33.144968 9940 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4, bc2c297fb28f4950864f0487ef83e833; no voters:
I20260808 13:36:33.145721 11054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260808 13:36:33.146044 11054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:33.146359 11054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 3 FOLLOWER]: Advancing to term 4
I20260808 13:36:33.156512 11054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:33.160466 11054 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 4 election: Requested vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:33.162943 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc2c297fb28f4950864f0487ef83e833"
I20260808 13:36:33.163513 9850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 3 FOLLOWER]: Advancing to term 4
W20260808 13:36:33.163715 9939 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:36:33.167912 9939 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:33.175484 9850 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 026dcc55447e4441b062e327f64eacb4 in term 4.
I20260808 13:36:33.177922 9940 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4, bc2c297fb28f4950864f0487ef83e833; no voters: 0bf9978620f543d8aee6279b9a9d3534
I20260808 13:36:33.179315 11054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 4 FOLLOWER]: Leader election won for term 4
I20260808 13:36:33.183918 11054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 4 LEADER]: Becoming Leader. State: Replica: 026dcc55447e4441b062e327f64eacb4, State: Running, Role: LEADER
I20260808 13:36:33.185120 11054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:33.192353 11057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 026dcc55447e4441b062e327f64eacb4. Latest consensus state: current_term: 4 leader_uuid: "026dcc55447e4441b062e327f64eacb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:33.193228 11057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:33.201422 11059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:36:33.210851 11059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:36:33.214912 11059 catalog_manager.cc:1259] Loaded cluster ID: 9eb2bfe6721b4b6487ca35bc443651ed
I20260808 13:36:33.215159 11059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:36:33.219028 11059 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:36:33.224160 11059 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Loaded TSK: 0
I20260808 13:36:33.225874 11059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:36:33.235056 11059 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260808 13:36:33.238811 11059 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
W20260808 13:36:33.240936 10492 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:33.267966 9958 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52746:
name: "default.Table_575a74e193bc45fd99dcd06ae9318532"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:33.270006 9961 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52746:
name: "default.Table_f7b46abc59bd4cacacf793d1d5a184b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:33.271476 9956 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52746:
name: "default.Table_82a5f186cbcd4cf0875234550bb2665e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:33.271780 9967 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52746:
name: "default.Table_e5e780904d7a4ad0a2b5ba7e99c1de0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:33.273609 10987 heartbeater.cc:503] Master 127.4.170.60:45415 was elected leader, sending a full tablet report...
W20260808 13:36:33.280452 10730 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:36:33.280966 10987 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:36:33.288893 9958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_575a74e193bc45fd99dcd06ae9318532 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:36:33.297178 9961 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f7b46abc59bd4cacacf793d1d5a184b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:36:33.297194 9956 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_82a5f186cbcd4cf0875234550bb2665e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 13:36:33.298872 9967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_e5e780904d7a4ad0a2b5ba7e99c1de0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:36:33.497622 9956 catalog_manager.cc:2148] created HMS catalog entry for table [id=1c41141d42bb4166b8a3a461f504c61b]
I20260808 13:36:33.506562 9850 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Refusing update from remote peer 026dcc55447e4441b062e327f64eacb4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260808 13:36:33.508441 11054 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260808 13:36:33.509117 9939 consensus_peers.cc:487] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 -> Peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Couldn't send request to peer 0bf9978620f543d8aee6279b9a9d3534. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 13:36:33.532346 11067 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 026dcc55447e4441b062e327f64eacb4. Latest consensus state: current_term: 4 leader_uuid: "026dcc55447e4441b062e327f64eacb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:33.533293 11067 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:33.538271 11066 mvcc.cc:205] Tried to move back new op lower bound from 7316259608585396224 to 7316259607304257536. Current Snapshot: MvccSnapshot[applied={T|T < 7316259608585396224}]
I20260808 13:36:33.545104 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.60:45415
--webserver_interface=127.4.170.60
--webserver_port=40025
--builtin_ntp_servers=127.4.170.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--hive_metastore_uris=thrift://127.0.0.1:46603
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260808 13:36:33.557217 11067 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "026dcc55447e4441b062e327f64eacb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:33.558022 11067 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:33.566799 11057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "026dcc55447e4441b062e327f64eacb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:33.567677 11057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:33.594421 9956 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52746:
table { table_name: "default.Table_82a5f186cbcd4cf0875234550bb2665e" } new_table_name: "default.Table_14527b1399a7419c90db6dbec907a64b" modify_external_catalogs: true
I20260808 13:36:33.654650 10396 tablet_service.cc:1467] Processing CreateTablet for tablet 7426fee1d9dd43b7b02c73e683135c90 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:33.656893 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7426fee1d9dd43b7b02c73e683135c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:33.660698 10868 tablet_service.cc:1467] Processing CreateTablet for tablet 7426fee1d9dd43b7b02c73e683135c90 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:33.663160 10868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7426fee1d9dd43b7b02c73e683135c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:33.669602 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 7426fee1d9dd43b7b02c73e683135c90 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:33.672640 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7426fee1d9dd43b7b02c73e683135c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:33.750840 11088 tablet_bootstrap.cc:492] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:33.754639 10731 heartbeater.cc:503] Master 127.4.170.61:37949 was elected leader, sending a full tablet report...
I20260808 13:36:33.764225 11089 tablet_bootstrap.cc:492] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:33.768200 10989 heartbeater.cc:503] Master 127.4.170.61:37949 was elected leader, sending a full tablet report...
I20260808 13:36:33.774991 11088 tablet_bootstrap.cc:654] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:33.775887 9951 master_service.cc:946] ReplaceTablet: received request to replace tablet 7426fee1d9dd43b7b02c73e683135c90 from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:33.777949 9950 master_service.cc:946] ReplaceTablet: received request to replace tablet 7426fee1d9dd43b7b02c73e683135c90 from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:33.794377 9948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52746:
name: "default.Table_6a12fa2b84814dcbb515f51589cbf401"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:33.800798 11088 log.cc:824] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:36:33.815660 9951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Sending DeleteTablet for 3 replicas of tablet 7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:33.818576 10396 tablet_service.cc:1514] Processing DeleteTablet for tablet 7426fee1d9dd43b7b02c73e683135c90 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f31bdd6c130465c990f446c70f2b3ec) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:33.818851 10868 tablet_service.cc:1514] Processing DeleteTablet for tablet 7426fee1d9dd43b7b02c73e683135c90 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f31bdd6c130465c990f446c70f2b3ec) from {username='slave'} at 127.0.0.1:49860
I20260808 13:36:33.821601 11089 tablet_bootstrap.cc:654] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:33.824834 11089 log.cc:824] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37: Log is configured to *not* fsync() on all Append() calls
W20260808 13:36:33.841048 9940 catalog_manager.cc:4821] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): delete failed for tablet 7426fee1d9dd43b7b02c73e683135c90 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7426fee1d9dd43b7b02c73e683135c90 already in progress: creating tablet
W20260808 13:36:33.844626 9940 catalog_manager.cc:4821] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): delete failed for tablet 7426fee1d9dd43b7b02c73e683135c90 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7426fee1d9dd43b7b02c73e683135c90 already in progress: creating tablet
I20260808 13:36:33.845266 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet 7426fee1d9dd43b7b02c73e683135c90 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f31bdd6c130465c990f446c70f2b3ec) from {username='slave'} at 127.0.0.1:41816
I20260808 13:36:33.857199 10396 tablet_service.cc:1467] Processing CreateTablet for tablet 7f31bdd6c130465c990f446c70f2b3ec (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:33.858552 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f31bdd6c130465c990f446c70f2b3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:33.861775 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 7f31bdd6c130465c990f446c70f2b3ec (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:33.863238 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f31bdd6c130465c990f446c70f2b3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:33.863638 11088 tablet_bootstrap.cc:492] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:33.864427 11088 ts_tablet_manager.cc:1397] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.114s user 0.012s sys 0.006s
I20260808 13:36:33.871637 9951 catalog_manager.cc:6565] ReplaceTablet: tablet 7426fee1d9dd43b7b02c73e683135c90 deleted and replaced by tablet 7f31bdd6c130465c990f446c70f2b3ec
I20260808 13:36:33.877404 10868 tablet_service.cc:1467] Processing CreateTablet for tablet 7f31bdd6c130465c990f446c70f2b3ec (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:33.879001 10868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f31bdd6c130465c990f446c70f2b3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:33.886804 11089 tablet_bootstrap.cc:492] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:33.887933 11089 ts_tablet_manager.cc:1397] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.124s user 0.014s sys 0.015s
W20260808 13:36:33.904222 9934 catalog_manager.cc:4821] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): delete failed for tablet 7426fee1d9dd43b7b02c73e683135c90 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7426fee1d9dd43b7b02c73e683135c90 already in progress: creating tablet
I20260808 13:36:33.907613 11102 tablet_bootstrap.cc:492] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:33.928539 11088 raft_consensus.cc:348] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } }
I20260808 13:36:33.929489 11088 raft_consensus.cc:374] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:33.929767 11088 raft_consensus.cc:729] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:33.930682 11088 consensus_queue.cc:260] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } }
I20260808 13:36:33.938303 11102 tablet_bootstrap.cc:654] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:33.942536 11102 log.cc:824] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995: Log is configured to *not* fsync() on all Append() calls
I20260808 13:36:33.948668 11088 ts_tablet_manager.cc:1428] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.084s user 0.032s sys 0.023s
I20260808 13:36:33.949517 10494 heartbeater.cc:503] Master 127.4.170.61:37949 was elected leader, sending a full tablet report...
I20260808 13:36:33.952939 11088 tablet_bootstrap.cc:492] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:33.959622 11088 tablet_bootstrap.cc:654] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:33.966869 10396 tablet_service.cc:1514] Processing DeleteTablet for tablet 7426fee1d9dd43b7b02c73e683135c90 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f31bdd6c130465c990f446c70f2b3ec) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:33.967840 11100 tablet_replica.cc:330] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:33.968710 11100 raft_consensus.cc:2229] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:33.969539 11100 raft_consensus.cc:2258] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:33.973536 11100 ts_tablet_manager.cc:1905] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:33.975504 11102 tablet_bootstrap.cc:492] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:33.976136 11102 ts_tablet_manager.cc:1397] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.069s user 0.007s sys 0.013s
I20260808 13:36:33.981371 11088 tablet_bootstrap.cc:492] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:33.981834 11088 ts_tablet_manager.cc:1397] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.029s user 0.017s sys 0.007s
I20260808 13:36:33.981456 11089 raft_consensus.cc:348] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } }
I20260808 13:36:33.982415 11089 raft_consensus.cc:374] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:33.982820 11089 raft_consensus.cc:729] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:33.984959 11088 raft_consensus.cc:348] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:33.985615 11088 raft_consensus.cc:374] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:33.985872 11088 raft_consensus.cc:729] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:33.986528 11088 consensus_queue.cc:260] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:33.989852 11089 consensus_queue.cc:260] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } }
I20260808 13:36:33.991062 11088 ts_tablet_manager.cc:1428] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260808 13:36:34.023663 9951 master_service.cc:946] ReplaceTablet: received request to replace tablet 7426fee1d9dd43b7b02c73e683135c90 from {username='slave'} at 127.0.0.1:52746
13:36:34.028 [HMS - ERROR - pool-6-thread-2] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_e5e780904d7a4ad0a2b5ba7e99c1de0e already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy25.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
I20260808 13:36:34.033231 11089 ts_tablet_manager.cc:1428] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.145s user 0.049s sys 0.031s
I20260808 13:36:34.036860 11100 ts_tablet_manager.cc:1918] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:34.037333 11100 log.cc:1196] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:34.038322 11100 ts_tablet_manager.cc:1939] T 7426fee1d9dd43b7b02c73e683135c90 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:34.041513 11089 tablet_bootstrap.cc:492] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:34.041396 9940 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet 7426fee1d9dd43b7b02c73e683135c90 (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
W20260808 13:36:34.041715 9951 catalog_manager.cc:5328] Ignoring report from unknown tablet 7426fee1d9dd43b7b02c73e683135c90
W20260808 13:36:34.045220 9967 catalog_manager.cc:2144] Already present: failed to create HMS catalog entry for table [id=63baed85e80942d78180ce1b57e725b0]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_e5e780904d7a4ad0a2b5ba7e99c1de0e already exists)
I20260808 13:36:34.051229 11089 tablet_bootstrap.cc:654] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.063963 9954 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52746:
name: "default.Table_face27da876543c6b6151d51b3eef9e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:34.078986 11102 raft_consensus.cc:348] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } }
I20260808 13:36:34.080085 11102 raft_consensus.cc:374] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.083510 9967 master_service.cc:946] ReplaceTablet: received request to replace tablet 7f31bdd6c130465c990f446c70f2b3ec from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:34.080489 11102 raft_consensus.cc:729] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.099928 11102 consensus_queue.cc:260] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } }
I20260808 13:36:34.103904 11089 tablet_bootstrap.cc:492] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:34.104429 11089 ts_tablet_manager.cc:1397] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.063s user 0.011s sys 0.013s
I20260808 13:36:34.107553 11089 raft_consensus.cc:348] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.108465 11089 raft_consensus.cc:374] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.108850 11089 raft_consensus.cc:729] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.109638 11089 consensus_queue.cc:260] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.112277 11089 ts_tablet_manager.cc:1428] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
W20260808 13:36:34.118451 9939 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:45415, user_credentials={real_user=slave}} blocked reactor thread for 59494us
I20260808 13:36:34.133278 11102 ts_tablet_manager.cc:1428] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.157s user 0.030s sys 0.009s
I20260808 13:36:34.134254 11102 tablet_bootstrap.cc:492] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
W20260808 13:36:34.135243 9950 catalog_manager.cc:5328] Ignoring report from unknown tablet 7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:34.140110 11102 tablet_bootstrap.cc:654] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
W20260808 13:36:34.143821 10503 tablet.cc:2369] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:36:34.162838 11102 tablet_bootstrap.cc:492] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:34.163374 11102 ts_tablet_manager.cc:1397] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.029s user 0.016s sys 0.007s
W20260808 13:36:34.167362 9950 catalog_manager.cc:5328] Ignoring report from unknown tablet 7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:34.167546 11102 raft_consensus.cc:348] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.168460 11102 raft_consensus.cc:374] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.168795 11102 raft_consensus.cc:729] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.169584 11102 consensus_queue.cc:260] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.172452 11102 ts_tablet_manager.cc:1428] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260808 13:36:34.180218 9967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Sending DeleteTablet for 3 replicas of tablet 7f31bdd6c130465c990f446c70f2b3ec
I20260808 13:36:34.182029 9967 catalog_manager.cc:6565] ReplaceTablet: tablet 7f31bdd6c130465c990f446c70f2b3ec deleted and replaced by tablet ca99b29941da4beabac2d841612055b5
I20260808 13:36:34.182852 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f31bdd6c130465c990f446c70f2b3ec with delete_type TABLET_DATA_DELETED (replaced by tablet ca99b29941da4beabac2d841612055b5) from {username='slave'} at 127.0.0.1:41816
I20260808 13:36:34.183764 11103 tablet_replica.cc:330] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:34.184629 11103 raft_consensus.cc:2229] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:34.185184 11103 raft_consensus.cc:2258] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.188012 10396 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f31bdd6c130465c990f446c70f2b3ec with delete_type TABLET_DATA_DELETED (replaced by tablet ca99b29941da4beabac2d841612055b5) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:34.188772 11100 tablet_replica.cc:330] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:34.188992 10868 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f31bdd6c130465c990f446c70f2b3ec with delete_type TABLET_DATA_DELETED (replaced by tablet ca99b29941da4beabac2d841612055b5) from {username='slave'} at 127.0.0.1:49860
I20260808 13:36:34.189446 11100 raft_consensus.cc:2229] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:34.189936 11100 raft_consensus.cc:2258] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.190853 11099 tablet_replica.cc:330] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
I20260808 13:36:34.191711 11099 raft_consensus.cc:2229] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:34.192299 11099 raft_consensus.cc:2258] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.194366 11100 ts_tablet_manager.cc:1905] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:34.196827 11103 ts_tablet_manager.cc:1905] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:34.206022 11099 ts_tablet_manager.cc:1905] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:34.213879 9961 catalog_manager.cc:2148] created HMS catalog entry for table [id=d9ec6ad6315a4df99b9739a8b5b5f639]
W20260808 13:36:34.215561 10739 tablet.cc:2369] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:36:34.235627 9967 master_service.cc:946] ReplaceTablet: received request to replace tablet 7f31bdd6c130465c990f446c70f2b3ec from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:34.239265 11100 ts_tablet_manager.cc:1918] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:34.239754 11100 log.cc:1196] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/7f31bdd6c130465c990f446c70f2b3ec
I20260808 13:36:34.240546 11100 ts_tablet_manager.cc:1939] T 7f31bdd6c130465c990f446c70f2b3ec P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:34.234411 11099 ts_tablet_manager.cc:1918] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:34.243386 11099 log.cc:1196] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/7f31bdd6c130465c990f446c70f2b3ec
I20260808 13:36:34.244455 11099 ts_tablet_manager.cc:1939] T 7f31bdd6c130465c990f446c70f2b3ec P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:34.246258 9940 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet 7f31bdd6c130465c990f446c70f2b3ec (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
I20260808 13:36:34.248127 9940 catalog_manager.cc:4841] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): tablet 7f31bdd6c130465c990f446c70f2b3ec (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
I20260808 13:36:34.252043 10396 tablet_service.cc:1467] Processing CreateTablet for tablet ca99b29941da4beabac2d841612055b5 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:36:34.252821 10992 tablet.cc:2369] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:36:34.253250 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca99b29941da4beabac2d841612055b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.267772 10625 tablet_service.cc:1467] Processing CreateTablet for tablet ca99b29941da4beabac2d841612055b5 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.268980 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca99b29941da4beabac2d841612055b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.272047 11088 tablet_bootstrap.cc:492] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:34.278472 11088 tablet_bootstrap.cc:654] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.279547 10868 tablet_service.cc:1467] Processing CreateTablet for tablet ca99b29941da4beabac2d841612055b5 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.280704 10868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca99b29941da4beabac2d841612055b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.294446 11088 tablet_bootstrap.cc:492] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:34.294989 11088 ts_tablet_manager.cc:1397] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.003s
I20260808 13:36:34.297737 11088 raft_consensus.cc:348] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.298854 9961 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52746 (request call id 19) took 1029 ms. Trace:
I20260808 13:36:34.298542 11088 raft_consensus.cc:374] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.302663 11088 raft_consensus.cc:729] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.303521 11088 consensus_queue.cc:260] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.299227 9961 rpcz_store.cc:276] 0808 13:36:33.269239 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:36:33.269375 (+ 136us) service_pool.cc:227] Handling call
0808 13:36:33.299679 (+ 30304us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:36:33.306017 (+ 6338us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:36:34.213833 (+907816us) catalog_manager.cc:2147] Created new table in HMS catalog
0808 13:36:34.298093 (+ 84260us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:36:34.298462 (+ 369us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:36:34.298779 (+ 317us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:36:34.218480 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:36:34.219042 (+ 562us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:36:34.219071 (+ 29us) write_op.cc:435] Acquired schema lock
0808 13:36:34.219088 (+ 17us) tablet.cc:660] Decoding operations
0808 13:36:34.219411 (+ 323us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:36:34.219479 (+ 68us) write_op.cc:641] Partition lock acquired for write op
0808 13:36:34.219507 (+ 28us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:36:34.219721 (+ 214us) tablet.cc:699] Row locks acquired
0808 13:36:34.219742 (+ 21us) write_op.cc:260] PREPARE: finished
0808 13:36:34.219861 (+ 119us) write_op.cc:270] Start()
0808 13:36:34.219963 (+ 102us) write_op.cc:276] Timestamp: P: 1786196194219815 usec, L: 0
0808 13:36:34.219983 (+ 20us) op_driver.cc:331] REPLICATION: starting
0808 13:36:34.236620 (+ 16637us) log.cc:842] Serialized 432 byte log entry
0808 13:36:34.296095 (+ 59475us) op_driver.cc:444] REPLICATION: finished
0808 13:36:34.296645 (+ 550us) write_op.cc:301] APPLY: starting
0808 13:36:34.296706 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0808 13:36:34.296812 (+ 106us) tablet.cc:1368] finished BulkCheckPresence
0808 13:36:34.296904 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:36:34.297253 (+ 349us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:36:34.297412 (+ 159us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:36:34.297459 (+ 47us) write_op.cc:312] APPLY: finished
0808 13:36:34.297645 (+ 186us) log.cc:842] Serialized 28 byte log entry
0808 13:36:34.297791 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:36:34.297862 (+ 71us) write_op.cc:454] Released schema lock
0808 13:36:34.297893 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0808 13:36:34.298071 (+ 178us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:36:34.298098 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":737416,"Hive Metastore.run_cpu_time_us":718,"Hive Metastore.run_wall_time_us":167181,"spinlock_wait_cycles":52096,"child_traces":[["op",{"apply.queue_time_us":376,"apply.run_cpu_time_us":1607,"apply.run_wall_time_us":1609,"num_ops":2,"prepare.queue_time_us":3001,"prepare.run_cpu_time_us":2307,"prepare.run_wall_time_us":32363,"replication_time_us":76030}]]}
I20260808 13:36:34.316761 11088 ts_tablet_manager.cc:1428] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20260808 13:36:34.326107 11102 tablet_bootstrap.cc:492] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:34.331516 11089 tablet_bootstrap.cc:492] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:34.333093 11103 ts_tablet_manager.cc:1918] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:34.335420 11103 log.cc:1196] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/7f31bdd6c130465c990f446c70f2b3ec
I20260808 13:36:34.335767 9958 catalog_manager.cc:2148] created HMS catalog entry for table [id=aaf27748800049ed98dbb95f79b7af14]
I20260808 13:36:34.336686 11103 ts_tablet_manager.cc:1939] T 7f31bdd6c130465c990f446c70f2b3ec P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:34.333173 11102 tablet_bootstrap.cc:654] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.339895 9934 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet 7f31bdd6c130465c990f446c70f2b3ec (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
I20260808 13:36:34.342813 11089 tablet_bootstrap.cc:654] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.354315 11102 tablet_bootstrap.cc:492] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:34.354848 11102 ts_tablet_manager.cc:1397] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.011s
I20260808 13:36:34.355957 9961 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52746:
table { table_name: "default.Table_f7b46abc59bd4cacacf793d1d5a184b6" } new_table_name: "default.Table_7a5955f4f27e416a846a78664eb3987a" modify_external_catalogs: true
I20260808 13:36:34.357745 11102 raft_consensus.cc:348] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.362967 11089 tablet_bootstrap.cc:492] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:34.363469 11089 ts_tablet_manager.cc:1397] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.004s
I20260808 13:36:34.366194 11089 raft_consensus.cc:348] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.367045 11089 raft_consensus.cc:374] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.367424 11089 raft_consensus.cc:729] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.358587 11102 raft_consensus.cc:374] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.368860 11102 raft_consensus.cc:729] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.370064 10868 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f31bdd6c130465c990f446c70f2b3ec with delete_type TABLET_DATA_DELETED (replaced by tablet ca99b29941da4beabac2d841612055b5) from {username='slave'} at 127.0.0.1:49860
I20260808 13:36:34.369658 11102 consensus_queue.cc:260] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.370242 11089 consensus_queue.cc:260] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.372105 11102 ts_tablet_manager.cc:1428] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.017s user 0.004s sys 0.003s
I20260808 13:36:34.379491 11089 ts_tablet_manager.cc:1428] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
W20260808 13:36:34.380913 9940 catalog_manager.cc:4808] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): delete failed for tablet 7f31bdd6c130465c990f446c70f2b3ec because the tablet was not found. No further retry: Not found: Tablet not found: 7f31bdd6c130465c990f446c70f2b3ec
F20260808 13:36:34.390123 10218 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_82a5f186cbcd4cf0875234550bb2665e
*** Check failure stack trace: ***
I20260808 13:36:34.400230 9946 master_service.cc:946] ReplaceTablet: received request to replace tablet ca99b29941da4beabac2d841612055b5 from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:34.410085 9958 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52746 (request call id 18) took 1143 ms. Trace:
I20260808 13:36:34.410483 9958 rpcz_store.cc:276] 0808 13:36:33.266297 (+ 0us) service_pool.cc:170] Inserting onto call queue
0808 13:36:33.266569 (+ 272us) service_pool.cc:227] Handling call
0808 13:36:33.289578 (+ 23009us) catalog_manager.cc:2075] Acquired catalog manager lock
0808 13:36:33.306963 (+ 17385us) catalog_manager.cc:2121] Created new table and tablet info
0808 13:36:34.335710 (+1028747us) catalog_manager.cc:2147] Created new table in HMS catalog
0808 13:36:34.409330 (+ 73620us) catalog_manager.cc:2181] Wrote table and tablets to system table
0808 13:36:34.409718 (+ 388us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0808 13:36:34.410017 (+ 299us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0808 13:36:34.337883 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0808 13:36:34.338324 (+ 441us) write_op.cc:432] Acquiring schema lock in shared mode
0808 13:36:34.338349 (+ 25us) write_op.cc:435] Acquired schema lock
0808 13:36:34.338365 (+ 16us) tablet.cc:660] Decoding operations
0808 13:36:34.338749 (+ 384us) write_op.cc:620] Acquiring the partition lock for write op
0808 13:36:34.338875 (+ 126us) write_op.cc:641] Partition lock acquired for write op
0808 13:36:34.338908 (+ 33us) tablet.cc:683] Acquiring locks for 2 operations
0808 13:36:34.339152 (+ 244us) tablet.cc:699] Row locks acquired
0808 13:36:34.339170 (+ 18us) write_op.cc:260] PREPARE: finished
0808 13:36:34.339305 (+ 135us) write_op.cc:270] Start()
0808 13:36:34.339401 (+ 96us) write_op.cc:276] Timestamp: P: 1786196194339235 usec, L: 0
0808 13:36:34.339425 (+ 24us) op_driver.cc:331] REPLICATION: starting
0808 13:36:34.357324 (+ 17899us) log.cc:842] Serialized 432 byte log entry
0808 13:36:34.360122 (+ 2798us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b5400021554. stack: 00007f67348d4033 00007f67348d3f30 00007f6734455534 00000000004db2b4 00000000004db249 00007f673b7e0471 00007f673b857198 00007f673bcb8ade 00007f673bcb82b8 00007f673bcbcba1 00007f673bcbcb59 00007f673bcbcae9 00007f673bcbcab1 00007f673bcbbdad 0000000000580784 00000000005805b9
0808 13:36:34.407021 (+ 46899us) op_driver.cc:444] REPLICATION: finished
0808 13:36:34.407315 (+ 294us) write_op.cc:301] APPLY: starting
0808 13:36:34.407426 (+ 111us) tablet.cc:1365] starting BulkCheckPresence
0808 13:36:34.407557 (+ 131us) tablet.cc:1368] finished BulkCheckPresence
0808 13:36:34.407579 (+ 22us) tablet.cc:1370] starting ApplyRowOperation cycle
0808 13:36:34.408136 (+ 557us) tablet.cc:1381] finished ApplyRowOperation cycle
0808 13:36:34.408346 (+ 210us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0808 13:36:34.408417 (+ 71us) write_op.cc:312] APPLY: finished
0808 13:36:34.408683 (+ 266us) log.cc:842] Serialized 28 byte log entry
0808 13:36:34.408920 (+ 237us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:36:34.409019 (+ 99us) write_op.cc:454] Released schema lock
0808 13:36:34.409067 (+ 48us) write_op.cc:341] FINISH: Updating metrics
0808 13:36:34.409318 (+ 251us) write_op.cc:489] Releasing partition, row and schema locks
0808 13:36:34.409357 (+ 39us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":903805,"Hive Metastore.run_cpu_time_us":704,"Hive Metastore.run_wall_time_us":123972,"spinlock_wait_cycles":32512,"child_traces":[["op",{"apply.queue_time_us":99,"apply.run_cpu_time_us":2267,"apply.run_wall_time_us":2267,"num_ops":2,"prepare.queue_time_us":125,"prepare.run_cpu_time_us":5627,"prepare.run_wall_time_us":22515,"raft.queue_time_us":51634,"raft.run_cpu_time_us":983,"raft.run_wall_time_us":985,"replication_time_us":67513,"spinlock_wait_cycles":40053760}]]}
I20260808 13:36:34.425170 10625 tablet_service.cc:1467] Processing CreateTablet for tablet fbfc00f6a73b4972953ee28867a7ded4 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.441478 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbfc00f6a73b4972953ee28867a7ded4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.444720 10396 tablet_service.cc:1467] Processing CreateTablet for tablet fbfc00f6a73b4972953ee28867a7ded4 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.446055 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbfc00f6a73b4972953ee28867a7ded4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.449319 10868 tablet_service.cc:1467] Processing CreateTablet for tablet fbfc00f6a73b4972953ee28867a7ded4 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.450589 10868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbfc00f6a73b4972953ee28867a7ded4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.455521 9945 master_service.cc:946] ReplaceTablet: received request to replace tablet ca99b29941da4beabac2d841612055b5 from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:34.477771 11113 raft_consensus.cc:482] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:34.478443 11113 raft_consensus.cc:504] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } }
I20260808 13:36:34.496003 11113 leader_election.cc:288] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:34.526976 11089 tablet_bootstrap.cc:492] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:34.533689 11089 tablet_bootstrap.cc:654] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
@ 0x7f67327379a0 google::LogMessage::Flush() at ??:0
I20260808 13:36:34.536772 9958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52746:
table { table_name: "default.Table_575a74e193bc45fd99dcd06ae9318532" } modify_external_catalogs: true
I20260808 13:36:34.537634 9958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52746:
table { table_name: "default.Table_575a74e193bc45fd99dcd06ae9318532" } modify_external_catalogs: true
I20260808 13:36:34.538630 11088 tablet_bootstrap.cc:492] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:34.544536 11088 tablet_bootstrap.cc:654] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
@ 0x7f673273770f google::LogMessage::~LogMessage() at ??:0
I20260808 13:36:34.571657 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7426fee1d9dd43b7b02c73e683135c90" candidate_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995" is_pre_election: true
I20260808 13:36:34.572603 10657 raft_consensus.cc:2454] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0e40695b6c9401aa0c178f58d8e2f37 in term 0.
W20260808 13:36:34.575049 10796 leader_election.cc:341] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:34.576402 10791 leader_election.cc:302] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37, e9d3df921f204d8aa02da52aca33d995; no voters: 117d8c07fa254197969de761dd678aa6
I20260808 13:36:34.577627 11113 raft_consensus.cc:2781] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:34.577961 11113 raft_consensus.cc:482] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:34.578265 11113 raft_consensus.cc:3037] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:34.582556 9946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Sending DeleteTablet for 3 replicas of tablet ca99b29941da4beabac2d841612055b5
I20260808 13:36:34.584129 9946 catalog_manager.cc:6565] ReplaceTablet: tablet ca99b29941da4beabac2d841612055b5 deleted and replaced by tablet 79cee72b59554d13acb268e787195c3f
I20260808 13:36:34.585251 10867 tablet_service.cc:1514] Processing DeleteTablet for tablet ca99b29941da4beabac2d841612055b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 79cee72b59554d13acb268e787195c3f) from {username='slave'} at 127.0.0.1:49860
I20260808 13:36:34.585233 9946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52746:
name: "default.Table_328786206e564026aa1c27ae85cf90f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:34.585999 11099 tablet_replica.cc:330] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
I20260808 13:36:34.586764 11099 raft_consensus.cc:2229] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:34.587210 11099 raft_consensus.cc:2258] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.587338 10396 tablet_service.cc:1514] Processing DeleteTablet for tablet ca99b29941da4beabac2d841612055b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 79cee72b59554d13acb268e787195c3f) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:34.588070 11100 tablet_replica.cc:330] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:34.588714 11100 raft_consensus.cc:2229] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:34.589174 11100 raft_consensus.cc:2258] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.590442 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet ca99b29941da4beabac2d841612055b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 79cee72b59554d13acb268e787195c3f) from {username='slave'} at 127.0.0.1:41816
I20260808 13:36:34.591020 11099 ts_tablet_manager.cc:1905] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:34.591311 11103 tablet_replica.cc:330] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:34.591895 11100 ts_tablet_manager.cc:1905] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:34.592049 11103 raft_consensus.cc:2229] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:34.592499 11103 raft_consensus.cc:2258] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.595544 11103 ts_tablet_manager.cc:1905] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:34.601727 11113 raft_consensus.cc:504] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } }
W20260808 13:36:34.609437 10796 leader_election.cc:341] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:34.611387 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7426fee1d9dd43b7b02c73e683135c90" candidate_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995"
I20260808 13:36:34.612046 10657 raft_consensus.cc:3037] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:34.625947 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.627125 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.634732 11113 leader_election.cc:288] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 election: Requested vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:34.635645 11102 tablet_bootstrap.cc:492] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:34.636647 10396 tablet_service.cc:1467] Processing CreateTablet for tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.637751 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.641443 11102 tablet_bootstrap.cc:654] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.646090 10657 raft_consensus.cc:2454] T 7426fee1d9dd43b7b02c73e683135c90 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e40695b6c9401aa0c178f58d8e2f37 in term 1.
I20260808 13:36:34.647375 10791 leader_election.cc:302] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37, e9d3df921f204d8aa02da52aca33d995; no voters: 117d8c07fa254197969de761dd678aa6
I20260808 13:36:34.649226 11113 raft_consensus.cc:2781] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:34.650146 10867 tablet_service.cc:1467] Processing CreateTablet for tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.651744 10867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.657310 11100 ts_tablet_manager.cc:1918] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:34.657797 11100 log.cc:1196] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/ca99b29941da4beabac2d841612055b5
I20260808 13:36:34.658759 11102 tablet_bootstrap.cc:492] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:34.659205 11102 ts_tablet_manager.cc:1397] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.004s
I20260808 13:36:34.661640 11102 raft_consensus.cc:348] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.662348 11102 raft_consensus.cc:374] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.664354 11088 tablet_bootstrap.cc:492] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:34.664767 11088 ts_tablet_manager.cc:1397] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.126s user 0.008s sys 0.004s
I20260808 13:36:34.667728 9951 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52746:
name: "default.Table_e071104baa0d4c3baca39e9ac21c484f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:34.668474 11100 ts_tablet_manager.cc:1939] T ca99b29941da4beabac2d841612055b5 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:34.673822 11088 raft_consensus.cc:348] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.674896 11088 raft_consensus.cc:374] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.675238 11088 raft_consensus.cc:729] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.676095 11088 consensus_queue.cc:260] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.662590 11102 raft_consensus.cc:729] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.684679 11102 consensus_queue.cc:260] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.685863 9940 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet ca99b29941da4beabac2d841612055b5 (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
I20260808 13:36:34.687670 11103 ts_tablet_manager.cc:1918] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:34.688050 11103 log.cc:1196] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/ca99b29941da4beabac2d841612055b5
I20260808 13:36:34.688673 11103 ts_tablet_manager.cc:1939] T ca99b29941da4beabac2d841612055b5 P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:34.692153 11088 ts_tablet_manager.cc:1428] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.027s user 0.008s sys 0.001s
I20260808 13:36:34.696641 11113 raft_consensus.cc:686] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 LEADER]: Becoming Leader. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Running, Role: LEADER
I20260808 13:36:34.697654 11113 consensus_queue.cc:237] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } }
I20260808 13:36:34.713044 11089 tablet_bootstrap.cc:492] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:34.713493 11089 ts_tablet_manager.cc:1397] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.187s user 0.011s sys 0.000s
I20260808 13:36:34.716116 11089 raft_consensus.cc:348] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.716833 11089 raft_consensus.cc:374] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.717110 11089 raft_consensus.cc:729] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.717756 11089 consensus_queue.cc:260] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.723953 11089 ts_tablet_manager.cc:1428] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260808 13:36:34.730033 11102 ts_tablet_manager.cc:1428] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.071s user 0.002s sys 0.004s
I20260808 13:36:34.736685 11099 ts_tablet_manager.cc:1918] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:34.737077 11099 log.cc:1196] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/ca99b29941da4beabac2d841612055b5
W20260808 13:36:34.737380 9967 catalog_manager.cc:5328] Ignoring report from unknown tablet 7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:34.737715 11099 ts_tablet_manager.cc:1939] T ca99b29941da4beabac2d841612055b5 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:34.737972 11102 tablet_bootstrap.cc:492] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:34.739106 11088 tablet_bootstrap.cc:492] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:34.744903 11088 tablet_bootstrap.cc:654] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.746639 11102 tablet_bootstrap.cc:654] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.746547 9934 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet ca99b29941da4beabac2d841612055b5 (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
I20260808 13:36:34.749362 11089 tablet_bootstrap.cc:492] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:34.757913 11089 tablet_bootstrap.cc:654] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.762519 9940 catalog_manager.cc:4841] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): tablet ca99b29941da4beabac2d841612055b5 (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
I20260808 13:36:34.769666 11088 tablet_bootstrap.cc:492] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:34.770129 11088 ts_tablet_manager.cc:1397] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.031s user 0.007s sys 0.003s
I20260808 13:36:34.774495 11088 raft_consensus.cc:348] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.779171 11088 raft_consensus.cc:374] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.780407 11088 raft_consensus.cc:729] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.781075 11088 consensus_queue.cc:260] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
W20260808 13:36:34.781711 11075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:36:34.782509 11075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:36:34.783257 11075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 13:36:34.786293 11102 tablet_bootstrap.cc:492] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:34.786783 11102 ts_tablet_manager.cc:1397] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.049s user 0.010s sys 0.004s
I20260808 13:36:34.792750 11089 tablet_bootstrap.cc:492] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:34.793181 11089 ts_tablet_manager.cc:1397] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.044s user 0.013s sys 0.001s
I20260808 13:36:34.796048 11089 raft_consensus.cc:348] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.796775 11089 raft_consensus.cc:374] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.797050 11089 raft_consensus.cc:729] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.783275 11088 ts_tablet_manager.cc:1428] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.013s user 0.008s sys 0.000s
I20260808 13:36:34.798707 11102 raft_consensus.cc:348] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.799453 11102 raft_consensus.cc:374] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.799729 11102 raft_consensus.cc:729] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.802261 11089 consensus_queue.cc:260] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.800549 11102 consensus_queue.cc:260] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.809974 11102 ts_tablet_manager.cc:1428] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260808 13:36:34.814327 11107 raft_consensus.cc:482] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:34.814911 11107 raft_consensus.cc:504] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.815725 11089 ts_tablet_manager.cc:1428] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.022s user 0.008s sys 0.000s
I20260808 13:36:34.822964 11107 leader_election.cc:288] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
@ 0x7f673d7d2aea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260808 13:36:34.840144 9950 master_service.cc:946] ReplaceTablet: received request to replace tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:34.845513 9945 master_service.cc:946] ReplaceTablet: received request to replace tablet fbfc00f6a73b4972953ee28867a7ded4 from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:34.847028 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfc00f6a73b4972953ee28867a7ded4" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995" is_pre_election: true
I20260808 13:36:34.847741 10657 raft_consensus.cc:2454] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 0.
I20260808 13:36:34.849321 10313 leader_election.cc:302] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:34.850365 11107 raft_consensus.cc:2781] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:34.850766 11107 raft_consensus.cc:482] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:34.851089 11107 raft_consensus.cc:3037] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:34.857504 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 79cee72b59554d13acb268e787195c3f (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.858639 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79cee72b59554d13acb268e787195c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.861723 10867 tablet_service.cc:1467] Processing CreateTablet for tablet 79cee72b59554d13acb268e787195c3f (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.863440 10396 tablet_service.cc:1467] Processing CreateTablet for tablet 79cee72b59554d13acb268e787195c3f (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:34.864544 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79cee72b59554d13acb268e787195c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.866910 10867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79cee72b59554d13acb268e787195c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:34.881465 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfc00f6a73b4972953ee28867a7ded4" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" is_pre_election: true
I20260808 13:36:34.882337 10905 raft_consensus.cc:2454] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 0.
I20260808 13:36:34.885514 11107 raft_consensus.cc:504] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.888593 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfc00f6a73b4972953ee28867a7ded4" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37"
I20260808 13:36:34.888911 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfc00f6a73b4972953ee28867a7ded4" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995"
I20260808 13:36:34.889138 10905 raft_consensus.cc:3037] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:34.889490 10657 raft_consensus.cc:3037] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:34.895676 10905 raft_consensus.cc:2454] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 1.
I20260808 13:36:34.896981 10315 leader_election.cc:302] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e0e40695b6c9401aa0c178f58d8e2f37; no voters:
I20260808 13:36:34.900610 11089 tablet_bootstrap.cc:492] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:34.904290 10657 raft_consensus.cc:2454] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 1.
I20260808 13:36:34.906114 9950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Sending DeleteTablet for 3 replicas of tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68
I20260808 13:36:34.906522 11089 tablet_bootstrap.cc:654] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.908110 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6688b61e04ac4e2a942d167cf0519b0d) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:34.908900 11100 tablet_replica.cc:330] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:34.909580 11100 raft_consensus.cc:2229] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:34.910012 11100 raft_consensus.cc:2258] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.912739 11100 ts_tablet_manager.cc:1905] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:34.912882 11088 tablet_bootstrap.cc:492] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:34.915397 11107 leader_election.cc:288] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 election: Requested vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:34.915961 11107 raft_consensus.cc:2781] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:34.916430 11107 raft_consensus.cc:686] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 1 LEADER]: Becoming Leader. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Running, Role: LEADER
I20260808 13:36:34.917366 11107 consensus_queue.cc:237] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.919591 11088 tablet_bootstrap.cc:654] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.920133 9945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Sending DeleteTablet for 3 replicas of tablet fbfc00f6a73b4972953ee28867a7ded4
I20260808 13:36:34.921969 10867 tablet_service.cc:1514] Processing DeleteTablet for tablet fbfc00f6a73b4972953ee28867a7ded4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0151c4cb072438fa6dc59a3bfc046a5) from {username='slave'} at 127.0.0.1:49860
I20260808 13:36:34.922658 9950 catalog_manager.cc:6565] ReplaceTablet: tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 deleted and replaced by tablet 6688b61e04ac4e2a942d167cf0519b0d
I20260808 13:36:34.923378 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6688b61e04ac4e2a942d167cf0519b0d) from {username='slave'} at 127.0.0.1:41816
I20260808 13:36:34.924140 11103 tablet_replica.cc:330] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:34.924861 11103 raft_consensus.cc:2229] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:34.925328 11103 raft_consensus.cc:2258] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.926982 11099 tablet_replica.cc:330] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
I20260808 13:36:34.927718 11103 ts_tablet_manager.cc:1905] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:34.927767 11099 raft_consensus.cc:2229] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:34.928436 11099 raft_consensus.cc:2258] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.930574 11099 ts_tablet_manager.cc:1905] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:34.935390 10867 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6688b61e04ac4e2a942d167cf0519b0d) from {username='slave'} at 127.0.0.1:49860
I20260808 13:36:34.937515 11089 tablet_bootstrap.cc:492] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:34.937913 11089 ts_tablet_manager.cc:1397] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.038s user 0.007s sys 0.006s
I20260808 13:36:34.939652 9945 catalog_manager.cc:6565] ReplaceTablet: tablet fbfc00f6a73b4972953ee28867a7ded4 deleted and replaced by tablet d0151c4cb072438fa6dc59a3bfc046a5
I20260808 13:36:34.940277 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet fbfc00f6a73b4972953ee28867a7ded4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0151c4cb072438fa6dc59a3bfc046a5) from {username='slave'} at 127.0.0.1:41816
I20260808 13:36:34.940321 11089 raft_consensus.cc:348] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.940981 11089 raft_consensus.cc:374] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.941222 11089 raft_consensus.cc:729] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.941847 11089 consensus_queue.cc:260] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.943842 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet fbfc00f6a73b4972953ee28867a7ded4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0151c4cb072438fa6dc59a3bfc046a5) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:34.944437 10867 tablet_service.cc:1514] Processing DeleteTablet for tablet fbfc00f6a73b4972953ee28867a7ded4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0151c4cb072438fa6dc59a3bfc046a5) from {username='slave'} at 127.0.0.1:49860
@ 0x7f673d7d233c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260808 13:36:34.948024 11102 tablet_bootstrap.cc:492] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:34.954106 11103 ts_tablet_manager.cc:1918] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:34.954502 11103 log.cc:1196] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/1eac9d5a3d6647f1a566b3ab7d6aaa68
I20260808 13:36:34.954984 11102 tablet_bootstrap.cc:654] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:34.955185 11103 ts_tablet_manager.cc:1939] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:34.958663 11103 tablet_replica.cc:330] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:34.959344 11103 raft_consensus.cc:2229] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:34.959777 11103 raft_consensus.cc:2258] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.962548 11103 ts_tablet_manager.cc:1905] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:34.963590 11100 ts_tablet_manager.cc:1918] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:34.964006 11100 log.cc:1196] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/1eac9d5a3d6647f1a566b3ab7d6aaa68
I20260808 13:36:34.964519 11102 tablet_bootstrap.cc:492] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:34.964701 11100 ts_tablet_manager.cc:1939] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:34.964947 11102 ts_tablet_manager.cc:1397] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.009s
I20260808 13:36:34.958042 9934 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:34.967700 9940 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:34.969202 11100 tablet_replica.cc:330] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
W20260808 13:36:34.973133 10315 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.3:35889, user_credentials={real_user=slave}} blocked reactor thread for 76379us
I20260808 13:36:34.974598 11102 raft_consensus.cc:348] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.975404 11102 raft_consensus.cc:374] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.983003 11102 raft_consensus.cc:729] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.983776 11102 consensus_queue.cc:260] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.984607 11088 tablet_bootstrap.cc:492] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:34.985064 11088 ts_tablet_manager.cc:1397] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.072s user 0.011s sys 0.007s
I20260808 13:36:34.987466 11088 raft_consensus.cc:348] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:34.988158 11088 raft_consensus.cc:374] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:34.988477 11088 raft_consensus.cc:729] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:34.991098 11089 ts_tablet_manager.cc:1428] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.053s user 0.007s sys 0.001s
I20260808 13:36:34.994124 11099 ts_tablet_manager.cc:1918] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:34.994554 11099 log.cc:1196] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/fbfc00f6a73b4972953ee28867a7ded4
I20260808 13:36:34.995301 11099 ts_tablet_manager.cc:1939] T fbfc00f6a73b4972953ee28867a7ded4 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:34.995900 11100 raft_consensus.cc:2229] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:36:34.996786 11102 ts_tablet_manager.cc:1428] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.032s user 0.008s sys 0.000s
I20260808 13:36:35.001230 11100 pending_rounds.cc:62] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: Trying to abort 1 pending ops.
I20260808 13:36:35.001503 11100 pending_rounds.cc:68] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259614612488192 op_type: NO_OP noop_request { }
I20260808 13:36:35.001962 11100 raft_consensus.cc:2866] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:36:35.002338 11100 raft_consensus.cc:2258] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:34.989084 11088 consensus_queue.cc:260] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.009541 11088 ts_tablet_manager.cc:1428] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260808 13:36:35.021136 10396 tablet_service.cc:1467] Processing CreateTablet for tablet 6688b61e04ac4e2a942d167cf0519b0d (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.022006 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 6688b61e04ac4e2a942d167cf0519b0d (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.022286 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6688b61e04ac4e2a942d167cf0519b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.022545 10867 tablet_service.cc:1467] Processing CreateTablet for tablet 6688b61e04ac4e2a942d167cf0519b0d (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.023119 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6688b61e04ac4e2a942d167cf0519b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.023706 10867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6688b61e04ac4e2a942d167cf0519b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.032704 9940 catalog_manager.cc:4841] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): tablet fbfc00f6a73b4972953ee28867a7ded4 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]) successfully deleted
I20260808 13:36:35.033406 11099 tablet_replica.cc:330] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
I20260808 13:36:35.041752 11099 raft_consensus.cc:2229] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.042256 11099 raft_consensus.cc:2258] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:35.045003 11088 tablet_bootstrap.cc:492] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:35.048875 11099 ts_tablet_manager.cc:1905] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.050897 10395 tablet_service.cc:1467] Processing CreateTablet for tablet d0151c4cb072438fa6dc59a3bfc046a5 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.051373 11088 tablet_bootstrap.cc:654] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.052134 10395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0151c4cb072438fa6dc59a3bfc046a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.059554 11102 tablet_bootstrap.cc:492] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
W20260808 13:36:35.059087 11075 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260808 13:36:35.063367 11075 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260808 13:36:35.063751 11075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:36:35.064103 11075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:36:35.064452 11075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260808 13:36:35.064636 11103 ts_tablet_manager.cc:1918] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260808 13:36:35.064796 11075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:36:35.065047 11103 log.cc:1196] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/fbfc00f6a73b4972953ee28867a7ded4
I20260808 13:36:35.065203 11102 tablet_bootstrap.cc:654] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.065745 11103 ts_tablet_manager.cc:1939] T fbfc00f6a73b4972953ee28867a7ded4 P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
@ 0x7f673d7d5fd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260808 13:36:35.072088 11088 tablet_bootstrap.cc:492] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:35.072541 11088 ts_tablet_manager.cc:1397] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.028s user 0.013s sys 0.003s
I20260808 13:36:35.075192 11088 raft_consensus.cc:348] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.075992 11088 raft_consensus.cc:374] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.076223 11088 raft_consensus.cc:729] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.076970 11088 consensus_queue.cc:260] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.079183 11088 ts_tablet_manager.cc:1428] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 13:36:35.084961 10627 tablet_service.cc:1467] Processing CreateTablet for tablet d0151c4cb072438fa6dc59a3bfc046a5 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
W20260808 13:36:35.084740 11167 consensus_queue.cc:1159] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "117d8c07fa254197969de761dd678aa6" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260808 13:36:35.086161 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0151c4cb072438fa6dc59a3bfc046a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.091502 10868 tablet_service.cc:1467] Processing CreateTablet for tablet d0151c4cb072438fa6dc59a3bfc046a5 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.092654 10868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0151c4cb072438fa6dc59a3bfc046a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.092752 11100 ts_tablet_manager.cc:1905] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.106055 11089 tablet_bootstrap.cc:492] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:35.113956 11089 tablet_bootstrap.cc:654] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.117465 11088 tablet_bootstrap.cc:492] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:35.121086 9934 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet fbfc00f6a73b4972953ee28867a7ded4 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]) successfully deleted
I20260808 13:36:35.120369 11099 ts_tablet_manager.cc:1918] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.122917 11099 log.cc:1196] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/1eac9d5a3d6647f1a566b3ab7d6aaa68
I20260808 13:36:35.126825 11088 tablet_bootstrap.cc:654] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
W20260808 13:36:35.129431 9939 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.4.170.60:45415, user_credentials={real_user=slave}} blocked reactor thread for 83816.7us
I20260808 13:36:35.124058 11099 ts_tablet_manager.cc:1939] T 1eac9d5a3d6647f1a566b3ab7d6aaa68 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:35.125862 11089 tablet_bootstrap.cc:492] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:35.132275 11089 ts_tablet_manager.cc:1397] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.002s
I20260808 13:36:35.134531 11089 raft_consensus.cc:348] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.135181 11089 raft_consensus.cc:374] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.135421 11089 raft_consensus.cc:729] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.136049 11089 consensus_queue.cc:260] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.138029 9940 catalog_manager.cc:4841] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): tablet 1eac9d5a3d6647f1a566b3ab7d6aaa68 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:35.141451 9967 master_service.cc:946] ReplaceTablet: received request to replace tablet 6688b61e04ac4e2a942d167cf0519b0d from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:35.144944 11089 ts_tablet_manager.cc:1428] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
W20260808 13:36:35.146402 9940 catalog_manager.cc:4808] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): delete failed for tablet fbfc00f6a73b4972953ee28867a7ded4 because the tablet was not found. No further retry: Not found: Tablet not found: fbfc00f6a73b4972953ee28867a7ded4
I20260808 13:36:35.156924 11102 tablet_bootstrap.cc:492] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:35.157392 11102 ts_tablet_manager.cc:1397] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.098s user 0.016s sys 0.037s
I20260808 13:36:35.157537 11100 ts_tablet_manager.cc:1918] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:35.157953 11100 log.cc:1196] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/fbfc00f6a73b4972953ee28867a7ded4
I20260808 13:36:35.158689 11100 ts_tablet_manager.cc:1939] T fbfc00f6a73b4972953ee28867a7ded4 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:35.159912 11102 raft_consensus.cc:348] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.160115 11075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:46603
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.60:45415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.170.60
--webserver_port=40025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:36:35.160544 11102 raft_consensus.cc:374] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.161903 9940 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet fbfc00f6a73b4972953ee28867a7ded4 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]) successfully deleted
I20260808 13:36:35.162055 11075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:36:35.163292 11088 tablet_bootstrap.cc:492] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:35.163688 11088 ts_tablet_manager.cc:1397] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.003s
I20260808 13:36:35.164409 11075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 13:36:35.166181 11088 raft_consensus.cc:348] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.166903 11088 raft_consensus.cc:374] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.167017 11089 tablet_bootstrap.cc:492] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:35.167145 11088 raft_consensus.cc:729] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.167714 11088 consensus_queue.cc:260] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.169581 11088 ts_tablet_manager.cc:1428] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260808 13:36:35.173921 11089 tablet_bootstrap.cc:654] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.175061 9945 master_service.cc:946] ReplaceTablet: received request to replace tablet 79cee72b59554d13acb268e787195c3f from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:35.188120 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet fbfc00f6a73b4972953ee28867a7ded4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0151c4cb072438fa6dc59a3bfc046a5) from {username='slave'} at 127.0.0.1:40342
W20260808 13:36:35.189826 9940 catalog_manager.cc:4808] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): delete failed for tablet fbfc00f6a73b4972953ee28867a7ded4 because the tablet was not found. No further retry: Not found: Tablet not found: fbfc00f6a73b4972953ee28867a7ded4
I20260808 13:36:35.160781 11102 raft_consensus.cc:729] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.195401 11102 consensus_queue.cc:260] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.199064 11102 ts_tablet_manager.cc:1428] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.041s user 0.006s sys 0.001s
I20260808 13:36:35.205957 11102 tablet_bootstrap.cc:492] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
@ 0x7f673d7d5f8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260808 13:36:35.214912 9967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Sending DeleteTablet for 3 replicas of tablet 6688b61e04ac4e2a942d167cf0519b0d
I20260808 13:36:35.217219 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 6688b61e04ac4e2a942d167cf0519b0d with delete_type TABLET_DATA_DELETED (replaced by tablet 39172037294d4e369b3e54adca3f5c50) from {username='slave'} at 127.0.0.1:41816
I20260808 13:36:35.218279 9967 catalog_manager.cc:6565] ReplaceTablet: tablet 6688b61e04ac4e2a942d167cf0519b0d deleted and replaced by tablet 39172037294d4e369b3e54adca3f5c50
I20260808 13:36:35.221100 11102 tablet_bootstrap.cc:654] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.225673 11089 tablet_bootstrap.cc:492] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:35.226095 11089 ts_tablet_manager.cc:1397] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.059s user 0.014s sys 0.029s
I20260808 13:36:35.228418 10868 tablet_service.cc:1514] Processing DeleteTablet for tablet 6688b61e04ac4e2a942d167cf0519b0d with delete_type TABLET_DATA_DELETED (replaced by tablet 39172037294d4e369b3e54adca3f5c50) from {username='slave'} at 127.0.0.1:49860
I20260808 13:36:35.228574 11089 raft_consensus.cc:348] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.229172 11099 tablet_replica.cc:330] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
I20260808 13:36:35.229238 11089 raft_consensus.cc:374] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.229530 11089 raft_consensus.cc:729] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.229934 11099 raft_consensus.cc:2229] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.230382 11099 raft_consensus.cc:2258] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:35.230212 11089 consensus_queue.cc:260] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.232173 11089 ts_tablet_manager.cc:1428] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260808 13:36:35.232748 11099 ts_tablet_manager.cc:1905] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.239825 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet 6688b61e04ac4e2a942d167cf0519b0d with delete_type TABLET_DATA_DELETED (replaced by tablet 39172037294d4e369b3e54adca3f5c50) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:35.240806 11100 tablet_replica.cc:330] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:35.241804 11100 raft_consensus.cc:2229] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.242274 11100 raft_consensus.cc:2258] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:35.244481 11100 ts_tablet_manager.cc:1905] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.246681 11102 tablet_bootstrap.cc:492] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:35.248802 11102 ts_tablet_manager.cc:1397] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.043s user 0.010s sys 0.010s
I20260808 13:36:35.247946 11103 tablet_replica.cc:330] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:35.251276 11103 raft_consensus.cc:2229] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.251928 11103 raft_consensus.cc:2258] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:35.257164 11099 ts_tablet_manager.cc:1918] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.257566 11099 log.cc:1196] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/6688b61e04ac4e2a942d167cf0519b0d
I20260808 13:36:35.257966 9945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Sending DeleteTablet for 3 replicas of tablet 79cee72b59554d13acb268e787195c3f
I20260808 13:36:35.258266 11099 ts_tablet_manager.cc:1939] T 6688b61e04ac4e2a942d167cf0519b0d P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:35.258265 11103 ts_tablet_manager.cc:1905] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.259552 9945 catalog_manager.cc:6565] ReplaceTablet: tablet 79cee72b59554d13acb268e787195c3f deleted and replaced by tablet 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:36:35.262518 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 79cee72b59554d13acb268e787195c3f with delete_type TABLET_DATA_DELETED (replaced by tablet 3e117c2bf59245a89dad2c6e96bdcfb6) from {username='slave'} at 127.0.0.1:41816
I20260808 13:36:35.265477 10868 tablet_service.cc:1514] Processing DeleteTablet for tablet 79cee72b59554d13acb268e787195c3f with delete_type TABLET_DATA_DELETED (replaced by tablet 3e117c2bf59245a89dad2c6e96bdcfb6) from {username='slave'} at 127.0.0.1:49860
I20260808 13:36:35.265697 9940 catalog_manager.cc:4841] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): tablet 6688b61e04ac4e2a942d167cf0519b0d (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:35.266909 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet 79cee72b59554d13acb268e787195c3f with delete_type TABLET_DATA_DELETED (replaced by tablet 3e117c2bf59245a89dad2c6e96bdcfb6) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:35.267287 11102 raft_consensus.cc:348] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.268221 11102 raft_consensus.cc:374] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.268532 11102 raft_consensus.cc:729] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.269796 11102 consensus_queue.cc:260] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
W20260808 13:36:35.271900 11187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:36:35.276096 10982 debug-util.cc:398] Leaking SignalData structure 0x7b08000af9c0 after lost signal to thread 10747
I20260808 13:36:35.282845 11102 ts_tablet_manager.cc:1428] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20260808 13:36:35.291539 11100 ts_tablet_manager.cc:1918] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.291985 11100 log.cc:1196] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/6688b61e04ac4e2a942d167cf0519b0d
I20260808 13:36:35.292943 11099 tablet_replica.cc:330] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
I20260808 13:36:35.298508 11099 raft_consensus.cc:2229] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.299144 11099 raft_consensus.cc:2258] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Raft consensus is shut down!
@ 0x7f673d7d5f1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260808 13:36:35.300689 11118 raft_consensus.cc:482] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:35.301220 11118 raft_consensus.cc:504] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.312240 11118 leader_election.cc:288] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:35.292701 11100 ts_tablet_manager.cc:1939] T 6688b61e04ac4e2a942d167cf0519b0d P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:35.316921 11100 tablet_replica.cc:330] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:35.316841 9940 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet 6688b61e04ac4e2a942d167cf0519b0d (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:35.317575 11100 raft_consensus.cc:2229] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.318027 11100 raft_consensus.cc:2258] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:35.323433 11100 ts_tablet_manager.cc:1905] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.325888 11099 ts_tablet_manager.cc:1905] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.328365 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet 6688b61e04ac4e2a942d167cf0519b0d with delete_type TABLET_DATA_DELETED (replaced by tablet 39172037294d4e369b3e54adca3f5c50) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:35.331650 10395 tablet_service.cc:1467] Processing CreateTablet for tablet 39172037294d4e369b3e54adca3f5c50 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.332903 10395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39172037294d4e369b3e54adca3f5c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.345538 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 39172037294d4e369b3e54adca3f5c50 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.346812 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39172037294d4e369b3e54adca3f5c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.361299 10868 tablet_service.cc:1467] Processing CreateTablet for tablet 39172037294d4e369b3e54adca3f5c50 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.362658 10868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39172037294d4e369b3e54adca3f5c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.369424 11100 ts_tablet_manager.cc:1918] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.369839 11100 log.cc:1196] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/79cee72b59554d13acb268e787195c3f
I20260808 13:36:35.373669 10396 tablet_service.cc:1467] Processing CreateTablet for tablet 3e117c2bf59245a89dad2c6e96bdcfb6 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.374940 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e117c2bf59245a89dad2c6e96bdcfb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.370496 11100 ts_tablet_manager.cc:1939] T 79cee72b59554d13acb268e787195c3f P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:35.388578 11088 tablet_bootstrap.cc:492] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:35.390838 9940 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet 79cee72b59554d13acb268e787195c3f (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
W20260808 13:36:35.392352 9940 catalog_manager.cc:4808] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): delete failed for tablet 6688b61e04ac4e2a942d167cf0519b0d because the tablet was not found. No further retry: Not found: Tablet not found: 6688b61e04ac4e2a942d167cf0519b0d
I20260808 13:36:35.394006 11088 tablet_bootstrap.cc:654] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.395388 10867 tablet_service.cc:1467] Processing CreateTablet for tablet 3e117c2bf59245a89dad2c6e96bdcfb6 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.396659 10867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e117c2bf59245a89dad2c6e96bdcfb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.402776 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 3e117c2bf59245a89dad2c6e96bdcfb6 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.404107 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e117c2bf59245a89dad2c6e96bdcfb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.425799 11103 ts_tablet_manager.cc:1918] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.426437 11103 log.cc:1196] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/6688b61e04ac4e2a942d167cf0519b0d
I20260808 13:36:35.427393 11103 ts_tablet_manager.cc:1939] T 6688b61e04ac4e2a942d167cf0519b0d P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:35.429294 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0151c4cb072438fa6dc59a3bfc046a5" candidate_uuid: "e9d3df921f204d8aa02da52aca33d995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" is_pre_election: true
I20260808 13:36:35.429960 10905 raft_consensus.cc:2454] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9d3df921f204d8aa02da52aca33d995 in term 0.
I20260808 13:36:35.431576 10565 leader_election.cc:302] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:35.432574 11118 raft_consensus.cc:2781] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:35.432924 11118 raft_consensus.cc:482] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:35.433218 11118 raft_consensus.cc:3037] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:35.435174 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0151c4cb072438fa6dc59a3bfc046a5" candidate_uuid: "e9d3df921f204d8aa02da52aca33d995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d8c07fa254197969de761dd678aa6" is_pre_election: true
I20260808 13:36:35.435950 10423 raft_consensus.cc:2454] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9d3df921f204d8aa02da52aca33d995 in term 0.
I20260808 13:36:35.437742 9934 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet 6688b61e04ac4e2a942d167cf0519b0d (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:35.438197 11102 tablet_bootstrap.cc:492] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:35.438725 11103 tablet_replica.cc:330] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:35.439512 11103 raft_consensus.cc:2229] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.439980 11103 raft_consensus.cc:2258] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:35.441262 11118 raft_consensus.cc:504] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.443261 11118 leader_election.cc:288] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [CANDIDATE]: Term 1 election: Requested vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:35.445695 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0151c4cb072438fa6dc59a3bfc046a5" candidate_uuid: "e9d3df921f204d8aa02da52aca33d995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37"
I20260808 13:36:35.446367 10905 raft_consensus.cc:3037] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:35.448043 11103 ts_tablet_manager.cc:1905] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.450755 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0151c4cb072438fa6dc59a3bfc046a5" candidate_uuid: "e9d3df921f204d8aa02da52aca33d995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d8c07fa254197969de761dd678aa6"
I20260808 13:36:35.451598 10423 raft_consensus.cc:3037] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:35.452350 9950 master_service.cc:946] ReplaceTablet: received request to replace tablet d0151c4cb072438fa6dc59a3bfc046a5 from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:35.458142 10905 raft_consensus.cc:2454] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9d3df921f204d8aa02da52aca33d995 in term 1.
I20260808 13:36:35.459826 10565 leader_election.cc:302] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:35.443918 11102 tablet_bootstrap.cc:654] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.467764 11118 raft_consensus.cc:2781] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:35.469512 11099 ts_tablet_manager.cc:1918] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.473711 11099 log.cc:1196] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/79cee72b59554d13acb268e787195c3f
I20260808 13:36:35.477875 11102 tablet_bootstrap.cc:492] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:35.478313 11102 ts_tablet_manager.cc:1397] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.040s user 0.008s sys 0.003s
I20260808 13:36:35.480768 11102 raft_consensus.cc:348] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.481503 11102 raft_consensus.cc:374] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.481809 11102 raft_consensus.cc:729] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.481153 11099 ts_tablet_manager.cc:1939] T 79cee72b59554d13acb268e787195c3f P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:35.482959 9950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Sending DeleteTablet for 3 replicas of tablet d0151c4cb072438fa6dc59a3bfc046a5
I20260808 13:36:35.484936 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet d0151c4cb072438fa6dc59a3bfc046a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:35.485816 11100 tablet_replica.cc:330] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:35.487023 11102 consensus_queue.cc:260] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.487926 9950 catalog_manager.cc:6565] ReplaceTablet: tablet d0151c4cb072438fa6dc59a3bfc046a5 deleted and replaced by tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4
I20260808 13:36:35.488912 10866 tablet_service.cc:1514] Processing DeleteTablet for tablet d0151c4cb072438fa6dc59a3bfc046a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4) from {username='slave'} at 127.0.0.1:49860
I20260808 13:36:35.489917 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet d0151c4cb072438fa6dc59a3bfc046a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4) from {username='slave'} at 127.0.0.1:41816
I20260808 13:36:35.491859 10423 raft_consensus.cc:2454] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9d3df921f204d8aa02da52aca33d995 in term 1.
I20260808 13:36:35.492272 11100 raft_consensus.cc:2229] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.492992 11088 tablet_bootstrap.cc:492] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:35.493475 11088 ts_tablet_manager.cc:1397] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.105s user 0.005s sys 0.017s
I20260808 13:36:35.495695 11100 raft_consensus.cc:2258] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:35.495942 11088 raft_consensus.cc:348] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.496645 11088 raft_consensus.cc:374] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.496912 11088 raft_consensus.cc:729] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.497444 11088 consensus_queue.cc:260] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.497934 11100 ts_tablet_manager.cc:1905] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.500308 11088 ts_tablet_manager.cc:1428] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 13:36:35.503180 11103 ts_tablet_manager.cc:1918] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.503585 11103 log.cc:1196] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/79cee72b59554d13acb268e787195c3f
I20260808 13:36:35.504227 11103 ts_tablet_manager.cc:1939] T 79cee72b59554d13acb268e787195c3f P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:35.508127 9967 master_service.cc:946] ReplaceTablet: received request to replace tablet d0151c4cb072438fa6dc59a3bfc046a5 from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:35.514290 9934 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet 79cee72b59554d13acb268e787195c3f (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
I20260808 13:36:35.515609 11103 tablet_replica.cc:330] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:35.518370 11099 tablet_replica.cc:330] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
I20260808 13:36:35.518553 9940 catalog_manager.cc:4841] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): tablet 79cee72b59554d13acb268e787195c3f (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
@ 0x7f673d7d5ee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260808 13:36:35.527081 11099 raft_consensus.cc:2229] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.527698 11099 raft_consensus.cc:2258] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:35.530061 11099 ts_tablet_manager.cc:1905] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.535374 11102 ts_tablet_manager.cc:1428] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.057s user 0.012s sys 0.010s
I20260808 13:36:35.538698 11102 tablet_bootstrap.cc:492] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:35.541170 11088 tablet_bootstrap.cc:492] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:35.541708 11100 ts_tablet_manager.cc:1918] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.544348 11100 log.cc:1196] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/d0151c4cb072438fa6dc59a3bfc046a5
I20260808 13:36:35.544646 11102 tablet_bootstrap.cc:654] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.545043 11100 ts_tablet_manager.cc:1939] T d0151c4cb072438fa6dc59a3bfc046a5 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:35.546340 11088 tablet_bootstrap.cc:654] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.549259 9940 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet d0151c4cb072438fa6dc59a3bfc046a5 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]) successfully deleted
I20260808 13:36:35.549636 11089 tablet_bootstrap.cc:492] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:35.554531 11089 tablet_bootstrap.cc:654] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.567500 11118 raft_consensus.cc:686] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 1 LEADER]: Becoming Leader. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Running, Role: LEADER
I20260808 13:36:35.568616 11118 consensus_queue.cc:237] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.572466 11088 tablet_bootstrap.cc:492] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:35.574734 11088 ts_tablet_manager.cc:1397] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.034s user 0.011s sys 0.002s
I20260808 13:36:35.578091 11089 tablet_bootstrap.cc:492] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:35.578469 11089 ts_tablet_manager.cc:1397] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.003s
I20260808 13:36:35.600895 11102 tablet_bootstrap.cc:492] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:35.601334 11102 ts_tablet_manager.cc:1397] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.063s user 0.016s sys 0.000s
I20260808 13:36:35.601055 10396 tablet_service.cc:1467] Processing CreateTablet for tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.602226 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.603705 11102 raft_consensus.cc:348] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.604398 11102 raft_consensus.cc:374] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.604648 11102 raft_consensus.cc:729] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.605299 11102 consensus_queue.cc:260] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.609691 11089 raft_consensus.cc:348] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.610342 11089 raft_consensus.cc:374] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.610579 11089 raft_consensus.cc:729] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.613857 11099 ts_tablet_manager.cc:1918] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.614217 11099 log.cc:1196] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/d0151c4cb072438fa6dc59a3bfc046a5
I20260808 13:36:35.615377 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.616580 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260808 13:36:35.617075 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:36:35.618530 11099 ts_tablet_manager.cc:1939] T d0151c4cb072438fa6dc59a3bfc046a5 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:35.620154 11103 raft_consensus.cc:2229] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:36:35.621580 10867 tablet_service.cc:1467] Processing CreateTablet for tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.621871 9940 catalog_manager.cc:4841] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): tablet d0151c4cb072438fa6dc59a3bfc046a5 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]) successfully deleted
I20260808 13:36:35.622817 10867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.612483 11088 raft_consensus.cc:348] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.626852 11088 raft_consensus.cc:374] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.627173 11088 raft_consensus.cc:729] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.627823 11088 consensus_queue.cc:260] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.629004 11089 consensus_queue.cc:260] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.630955 11102 ts_tablet_manager.cc:1428] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.029s user 0.007s sys 0.001s
@ 0x7f673d7d51de std::__1::__function::__func<>::operator()() at ??:0
I20260808 13:36:35.646513 11103 pending_rounds.cc:62] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: Trying to abort 1 pending ops.
I20260808 13:36:35.646860 11103 pending_rounds.cc:68] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316259617162235904 op_type: NO_OP noop_request { }
I20260808 13:36:35.647496 11103 raft_consensus.cc:2866] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260808 13:36:35.647931 11103 raft_consensus.cc:2258] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
I20260808 13:36:35.653682 11089 ts_tablet_manager.cc:1428] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.075s user 0.005s sys 0.000s
I20260808 13:36:35.656913 11103 ts_tablet_manager.cc:1905] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.660543 11107 raft_consensus.cc:482] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:35.660957 11107 raft_consensus.cc:504] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.662750 11107 leader_election.cc:288] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:35.664318 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39172037294d4e369b3e54adca3f5c50" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" is_pre_election: true
I20260808 13:36:35.665271 10905 raft_consensus.cc:2454] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 0.
I20260808 13:36:35.667357 10315 leader_election.cc:302] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e0e40695b6c9401aa0c178f58d8e2f37; no voters:
I20260808 13:36:35.670176 11107 raft_consensus.cc:2781] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:35.670471 11107 raft_consensus.cc:482] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:35.670768 11107 raft_consensus.cc:3037] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:35.672415 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39172037294d4e369b3e54adca3f5c50" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995" is_pre_election: true
I20260808 13:36:35.673050 10657 raft_consensus.cc:2454] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 0.
@ 0x5805ba std::__1::function<>::operator()() at ??:?
I20260808 13:36:35.682134 11107 raft_consensus.cc:504] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.683934 11107 leader_election.cc:288] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 election: Requested vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:35.684760 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39172037294d4e369b3e54adca3f5c50" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995"
I20260808 13:36:35.685299 10657 raft_consensus.cc:3037] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:35.685428 11088 ts_tablet_manager.cc:1428] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.110s user 0.004s sys 0.001s
I20260808 13:36:35.691053 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39172037294d4e369b3e54adca3f5c50" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37"
I20260808 13:36:35.691610 10905 raft_consensus.cc:3037] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:35.698184 9950 master_service.cc:946] ReplaceTablet: received request to replace tablet 3e117c2bf59245a89dad2c6e96bdcfb6 from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:35.700745 10657 raft_consensus.cc:2454] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 1.
I20260808 13:36:35.701897 10313 leader_election.cc:302] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:35.698532 10905 raft_consensus.cc:2454] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 1.
I20260808 13:36:35.705156 11107 raft_consensus.cc:2781] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:35.711602 11103 ts_tablet_manager.cc:1918] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:35.712003 11103 log.cc:1196] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/d0151c4cb072438fa6dc59a3bfc046a5
I20260808 13:36:35.712751 11103 ts_tablet_manager.cc:1939] T d0151c4cb072438fa6dc59a3bfc046a5 P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:35.732394 11089 tablet_bootstrap.cc:492] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:35.756042 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e117c2bf59245a89dad2c6e96bdcfb6 with delete_type TABLET_DATA_DELETED (replaced by tablet bdb23f24298544528289f064911e0d16) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:35.756836 11100 tablet_replica.cc:330] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:35.757442 11100 raft_consensus.cc:2229] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.757905 11100 raft_consensus.cc:2258] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:35.758955 9934 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet d0151c4cb072438fa6dc59a3bfc046a5 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]) successfully deleted
I20260808 13:36:35.761452 11100 ts_tablet_manager.cc:1905] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.766779 9950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Sending DeleteTablet for 3 replicas of tablet 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:36:35.768813 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e117c2bf59245a89dad2c6e96bdcfb6 with delete_type TABLET_DATA_DELETED (replaced by tablet bdb23f24298544528289f064911e0d16) from {username='slave'} at 127.0.0.1:40342
I20260808 13:36:35.771067 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e117c2bf59245a89dad2c6e96bdcfb6 with delete_type TABLET_DATA_DELETED (replaced by tablet bdb23f24298544528289f064911e0d16) from {username='slave'} at 127.0.0.1:41816
I20260808 13:36:35.771921 11103 tablet_replica.cc:330] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:35.772975 11103 raft_consensus.cc:2229] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:35.773600 11103 raft_consensus.cc:2258] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:35.775810 11103 ts_tablet_manager.cc:1905] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:35.792443 10868 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e117c2bf59245a89dad2c6e96bdcfb6 with delete_type TABLET_DATA_DELETED (replaced by tablet bdb23f24298544528289f064911e0d16) from {username='slave'} at 127.0.0.1:49860
I20260808 13:36:35.793087 9950 catalog_manager.cc:6565] ReplaceTablet: tablet 3e117c2bf59245a89dad2c6e96bdcfb6 deleted and replaced by tablet bdb23f24298544528289f064911e0d16
W20260808 13:36:35.793991 9940 catalog_manager.cc:4821] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): delete failed for tablet 3e117c2bf59245a89dad2c6e96bdcfb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3e117c2bf59245a89dad2c6e96bdcfb6 already in progress: creating tablet
I20260808 13:36:35.809232 11089 tablet_bootstrap.cc:654] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.811164 11107 raft_consensus.cc:686] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 1 LEADER]: Becoming Leader. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Running, Role: LEADER
I20260808 13:36:35.812858 11107 consensus_queue.cc:237] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
@ 0x7f67348e8dc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260808 13:36:35.829095 11088 tablet_bootstrap.cc:492] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:35.837673 11100 ts_tablet_manager.cc:1918] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.838027 11100 log.cc:1196] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:36:35.841877 9967 master_service.cc:946] ReplaceTablet: received request to replace tablet 3e117c2bf59245a89dad2c6e96bdcfb6 from {username='slave'} at 127.0.0.1:52746
I20260808 13:36:35.846993 11102 tablet_bootstrap.cc:492] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:35.852440 11102 tablet_bootstrap.cc:654] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.855597 11100 ts_tablet_manager.cc:1939] T 3e117c2bf59245a89dad2c6e96bdcfb6 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:35.856542 11103 ts_tablet_manager.cc:1918] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:35.856904 11103 log.cc:1196] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:36:35.857501 11103 ts_tablet_manager.cc:1939] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:35.860414 11113 consensus_queue.cc:1035] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:35.862049 9940 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet 3e117c2bf59245a89dad2c6e96bdcfb6 (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
W20260808 13:36:35.862898 9940 catalog_manager.cc:4808] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): delete failed for tablet 3e117c2bf59245a89dad2c6e96bdcfb6 because the tablet was not found. No further retry: Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:36:35.868546 11102 tablet_bootstrap.cc:492] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:35.869159 11102 ts_tablet_manager.cc:1397] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.000s
I20260808 13:36:35.844331 11088 tablet_bootstrap.cc:654] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.876483 10396 tablet_service.cc:1467] Processing CreateTablet for tablet bdb23f24298544528289f064911e0d16 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.877614 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdb23f24298544528289f064911e0d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.877974 9950 catalog_manager.cc:5495] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 117d8c07fa254197969de761dd678aa6 (127.4.170.1). New cstate: current_term: 1 leader_uuid: "117d8c07fa254197969de761dd678aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } health_report { overall_health: UNKNOWN } } }
@ 0x7f6735f566db start_thread at ??:0
I20260808 13:36:35.897171 9934 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet 3e117c2bf59245a89dad2c6e96bdcfb6 (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
W20260808 13:36:35.899791 9967 catalog_manager.cc:5328] Ignoring report from unknown tablet 7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:35.879056 11102 raft_consensus.cc:348] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.904481 11102 raft_consensus.cc:374] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.904886 11102 raft_consensus.cc:729] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.921134 11089 tablet_bootstrap.cc:492] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:35.921672 11089 ts_tablet_manager.cc:1397] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.189s user 0.020s sys 0.009s
I20260808 13:36:35.923288 10625 tablet_service.cc:1467] Processing CreateTablet for tablet bdb23f24298544528289f064911e0d16 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.924476 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdb23f24298544528289f064911e0d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.926559 10867 tablet_service.cc:1467] Processing CreateTablet for tablet bdb23f24298544528289f064911e0d16 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:35.927634 10867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdb23f24298544528289f064911e0d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:35.928146 11102 consensus_queue.cc:260] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.935523 11088 tablet_bootstrap.cc:492] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:35.935951 11088 ts_tablet_manager.cc:1397] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.108s user 0.009s sys 0.000s
@ 0x7f67313dc61f clone at ??:0
I20260808 13:36:35.940307 11088 raft_consensus.cc:348] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.940954 11088 raft_consensus.cc:374] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.941181 11088 raft_consensus.cc:729] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.942503 11089 raft_consensus.cc:348] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.943198 11089 raft_consensus.cc:374] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:35.943418 11089 raft_consensus.cc:729] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:35.943969 11089 consensus_queue.cc:260] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.945691 11089 ts_tablet_manager.cc:1428] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.024s user 0.003s sys 0.000s
I20260808 13:36:35.946499 11089 tablet_bootstrap.cc:492] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:35.955861 11088 consensus_queue.cc:260] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:35.969877 11089 tablet_bootstrap.cc:654] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:35.977541 11088 ts_tablet_manager.cc:1428] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.041s user 0.006s sys 0.001s
I20260808 13:36:35.994302 11089 tablet_bootstrap.cc:492] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:35.994764 11089 ts_tablet_manager.cc:1397] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.048s user 0.008s sys 0.003s
I20260808 13:36:35.994225 11088 tablet_bootstrap.cc:492] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:36.002578 11102 ts_tablet_manager.cc:1428] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.133s user 0.009s sys 0.000s
I20260808 13:36:36.002444 11089 raft_consensus.cc:348] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.003122 11089 raft_consensus.cc:374] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:36.003356 11089 raft_consensus.cc:729] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:36.006872 11089 consensus_queue.cc:260] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.012207 11088 tablet_bootstrap.cc:654] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:36.016494 11089 ts_tablet_manager.cc:1428] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260808 13:36:36.025576 11089 tablet_bootstrap.cc:492] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:36.026098 11102 tablet_bootstrap.cc:492] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:36.030776 11089 tablet_bootstrap.cc:654] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:36.054649 11107 raft_consensus.cc:482] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:36.055204 11107 raft_consensus.cc:504] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.057245 11107 leader_election.cc:288] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:36.057463 11089 tablet_bootstrap.cc:492] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:36.057843 11089 ts_tablet_manager.cc:1397] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.000s
I20260808 13:36:36.059461 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2ec07c8108414f8a27a5a3c3c1cbc4" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995" is_pre_election: true
I20260808 13:36:36.060120 10657 raft_consensus.cc:2454] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 0.
I20260808 13:36:36.060477 11102 tablet_bootstrap.cc:654] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:36.060019 11089 raft_consensus.cc:348] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.060772 11089 raft_consensus.cc:374] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:36.061008 11089 raft_consensus.cc:729] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:36.060951 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2ec07c8108414f8a27a5a3c3c1cbc4" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" is_pre_election: true
I20260808 13:36:36.061694 10905 raft_consensus.cc:2454] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 0.
I20260808 13:36:36.061869 11089 consensus_queue.cc:260] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.063299 10313 leader_election.cc:302] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:36.064604 11107 raft_consensus.cc:2781] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:36.064929 11107 raft_consensus.cc:482] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:36.065265 11107 raft_consensus.cc:3037] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:36.072381 11088 tablet_bootstrap.cc:492] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:36.072120 11107 raft_consensus.cc:504] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.072817 11088 ts_tablet_manager.cc:1397] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.079s user 0.007s sys 0.001s
I20260808 13:36:36.074081 11089 ts_tablet_manager.cc:1428] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260808 13:36:36.074930 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2ec07c8108414f8a27a5a3c3c1cbc4" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995"
I20260808 13:36:36.075567 10657 raft_consensus.cc:3037] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:36.075873 11088 raft_consensus.cc:348] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.076593 11088 raft_consensus.cc:374] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:36.076910 11088 raft_consensus.cc:729] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:36.082527 10657 raft_consensus.cc:2454] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 1.
I20260808 13:36:36.084088 10313 leader_election.cc:302] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:36.085362 11211 raft_consensus.cc:2781] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:36.087332 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2ec07c8108414f8a27a5a3c3c1cbc4" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37"
I20260808 13:36:36.087890 10905 raft_consensus.cc:3037] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:36.094442 10905 raft_consensus.cc:2454] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 1.
I20260808 13:36:36.096690 11107 leader_election.cc:288] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 election: Requested vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:36.104538 11088 consensus_queue.cc:260] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.105532 11107 raft_consensus.cc:482] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:36.105975 11107 raft_consensus.cc:504] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.107788 11088 ts_tablet_manager.cc:1428] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20260808 13:36:36.110497 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdb23f24298544528289f064911e0d16" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995" is_pre_election: true
W20260808 13:36:36.111927 10313 leader_election.cc:341] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:36.114040 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdb23f24298544528289f064911e0d16" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" is_pre_election: true
I20260808 13:36:36.114874 10905 raft_consensus.cc:2454] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 0.
I20260808 13:36:36.116415 10315 leader_election.cc:302] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e0e40695b6c9401aa0c178f58d8e2f37; no voters: e9d3df921f204d8aa02da52aca33d995
I20260808 13:36:36.117754 11107 leader_election.cc:288] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:36.118379 11238 raft_consensus.cc:2781] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:36.118774 11238 raft_consensus.cc:482] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:36.119210 11238 raft_consensus.cc:3037] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:36.131886 11102 tablet_bootstrap.cc:492] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:36.132337 11102 ts_tablet_manager.cc:1397] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.106s user 0.012s sys 0.000s
I20260808 13:36:36.134948 11102 raft_consensus.cc:348] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.135674 11102 raft_consensus.cc:374] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:36.135937 11102 raft_consensus.cc:729] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:36.136658 11102 consensus_queue.cc:260] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.138716 11102 ts_tablet_manager.cc:1428] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260808 13:36:36.147908 11211 raft_consensus.cc:686] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 1 LEADER]: Becoming Leader. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Running, Role: LEADER
I20260808 13:36:36.148871 11211 consensus_queue.cc:237] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.181173 11238 raft_consensus.cc:504] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.213459 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdb23f24298544528289f064911e0d16" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995"
I20260808 13:36:36.214120 10657 raft_consensus.cc:3037] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:36.222558 10657 raft_consensus.cc:2454] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 1.
I20260808 13:36:36.223839 10313 leader_election.cc:302] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:36.224789 11211 raft_consensus.cc:2781] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:36.226097 11211 raft_consensus.cc:686] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 1 LEADER]: Becoming Leader. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Running, Role: LEADER
I20260808 13:36:36.229887 11211 consensus_queue.cc:237] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.235560 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdb23f24298544528289f064911e0d16" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37"
I20260808 13:36:36.236214 10905 raft_consensus.cc:3037] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:36.245121 10905 raft_consensus.cc:2454] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 1.
I20260808 13:36:36.246968 11238 leader_election.cc:288] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 election: Requested vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:36.413758 11223 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:36.414336 11223 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:36.416563 11223 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:36:36.421739 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:36:36.422804 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:36:36.423177 10796 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:36:36.424177 11223 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:36.443760 11249 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 026dcc55447e4441b062e327f64eacb4)
I20260808 13:36:36.445466 11249 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:36.459719 11249 leader_election.cc:288] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:36.478782 9809 leader_election.cc:334] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:36:36.636757 10559 connection.cc:535] server connection from 127.0.0.1:41816 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:36.637853 11005 connection.cc:535] client connection to 127.4.170.61:37949 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:36.640390 11005 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.170.61:37949)
W20260808 13:36:36.641162 11005 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.4.170.61:37949)
W20260808 13:36:36.642447 11005 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.4.170.61:37949)
W20260808 13:36:36.646940 11005 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.4.170.61:37949)
W20260808 13:36:36.650914 10559 connection.cc:535] client connection to 127.4.170.61:37949 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:36.650955 10789 connection.cc:535] client connection to 127.4.170.61:37949 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:36.663277 10789 connection.cc:535] server connection from 127.0.0.1:49860 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:36.664278 10311 connection.cc:535] client connection to 127.4.170.61:37949 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:36.669308 9805 connection.cc:535] client connection to 127.4.170.61:37949 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:36.675753 11256 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:36:36.652835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.170.61:37949 (local address 127.4.170.3:55871)
0808 13:36:36.653935 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0808 13:36:36.654007 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 13:36:36.654456 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:36:36.674536 (+ 20080us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":368}
W20260808 13:36:36.678179 11160 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:36:36.666504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.170.61:37949 (local address 127.4.170.1:54965)
0808 13:36:36.667508 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0808 13:36:36.667567 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 13:36:36.668010 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:36:36.677082 (+ 9072us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":412}
W20260808 13:36:36.678679 11260 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:36:36.653093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.170.61:37949 (local address 127.4.170.2:47137)
0808 13:36:36.662301 (+ 9208us) negotiation.cc:107] Waiting for socket to connect
0808 13:36:36.662368 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 13:36:36.662664 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:36:36.677137 (+ 14473us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":8565,"thread_start_us":9499,"threads_started":1}
W20260808 13:36:36.681286 11251 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:36:36.671672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.170.61:37949 (local address 127.0.0.1:52772)
0808 13:36:36.672714 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0808 13:36:36.672758 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 13:36:36.673166 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:36:36.680160 (+ 6994us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":398}
W20260808 13:36:36.682225 9805 leader_election.cc:334] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949): Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 13:36:36.682651 9805 leader_election.cc:302] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc2c297fb28f4950864f0487ef83e833; no voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534
I20260808 13:36:36.683655 11249 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260808 13:36:36.705492 11257 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:36:36.679305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.170.61:37949 (local address 127.0.0.1:52762)
0808 13:36:36.694510 (+ 15205us) negotiation.cc:107] Waiting for socket to connect
0808 13:36:36.694801 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":14677,"thread_start_us":19603,"threads_started":1}
I20260808 13:36:36.919518 11265 consensus_queue.cc:1035] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260808 13:36:37.056450 11265 consensus_queue.cc:1035] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:36:37.122032 11273 consensus_queue.cc:1035] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260808 13:36:37.189038 11265 consensus_queue.cc:1035] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:36:37.332773 11289 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 026dcc55447e4441b062e327f64eacb4)
I20260808 13:36:37.333408 11289 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:37.343053 11289 leader_election.cc:288] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:37.357980 11270 consensus_queue.cc:1035] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:36:37.368381 9805 leader_election.cc:334] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949): Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:37.373669 11270 consensus_queue.cc:1035] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260808 13:36:37.411329 9809 leader_election.cc:334] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:37.413939 9809 leader_election.cc:302] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc2c297fb28f4950864f0487ef83e833; no voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534
I20260808 13:36:37.418457 11289 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260808 13:36:37.469956 9809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.60:45415, user_credentials={real_user=slave}} blocked reactor thread for 58677.7us
W20260808 13:36:36.625181 11181 debug-util.cc:398] Leaking SignalData structure 0x7b080001e820 after lost signal to thread 11075
W20260808 13:36:37.799099 11075 thread.cc:636] GCE (cloud detector) Time spent creating pthread: real 2.537s user 0.647s sys 0.925s
W20260808 13:36:37.805089 11075 thread.cc:602] GCE (cloud detector) Time spent starting thread: real 2.543s user 0.648s sys 0.925s
W20260808 13:36:35.293303 11185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:37.819686 11075 server_base.cc:1034] running on GCE node
W20260808 13:36:37.821214 11303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:37.823021 11075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:36:37.827772 11075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:36:37.829339 11075 hybrid_clock.cc:648] HybridClock initialized: now 1786196197829297 us; error 49 us; skew 500 ppm
I20260808 13:36:37.830452 11075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:36:37.844705 11075 webserver.cc:460] Webserver started at http://127.4.170.60:40025/ using document root <none> and password file <none>
I20260808 13:36:37.846120 11075 fs_manager.cc:359] Metadata directory not provided
I20260808 13:36:37.846459 11075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:36:37.859707 11075 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.007s sys 0.000s
I20260808 13:36:37.866163 11308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:37.867527 11075 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260808 13:36:37.867973 11075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal
uuid: "0bf9978620f543d8aee6279b9a9d3534"
format_stamp: "Formatted at 2026-08-08 13:36:24 on dist-test-slave-twwt"
I20260808 13:36:37.871868 11075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:36:37.919423 11075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:36:37.921504 11075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:36:37.922082 11075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:36:38.102878 11075 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.60:45415
I20260808 13:36:38.103035 11391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.60:45415 every 8 connection(s)
I20260808 13:36:38.107178 11075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-2/data/info.pb
I20260808 13:36:38.113185 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 11075
I20260808 13:36:38.120715 11393 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:36:38.139055 11393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Bootstrap starting.
I20260808 13:36:38.161172 11393 log.cc:824] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Log is configured to *not* fsync() on all Append() calls
I20260808 13:36:38.189339 11393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:36:38.190475 11393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Bootstrap complete.
I20260808 13:36:38.224643 11393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:38.228478 11393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf9978620f543d8aee6279b9a9d3534, State: Initialized, Role: FOLLOWER
I20260808 13:36:38.229523 11393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:38.232770 11405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:38.233704 11405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:38.234745 11393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:36:38.235847 10492 heartbeater.cc:347] Connected to a master server at 127.4.170.60:45415
I20260808 13:36:38.300117 11425 catalog_manager.cc:1259] Loaded cluster ID: 9eb2bfe6721b4b6487ca35bc443651ed
I20260808 13:36:38.300401 11425 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: loading cluster ID for follower catalog manager: success
I20260808 13:36:38.310536 11425 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: acquiring CA information for follower catalog manager: success
I20260808 13:36:38.350098 11425 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:36:38.350914 10987 heartbeater.cc:347] Connected to a master server at 127.4.170.60:45415
I20260808 13:36:38.351483 10987 heartbeater.cc:503] Master 127.4.170.60:45415 was elected leader, sending a full tablet report...
I20260808 13:36:38.346633 11075 master_runner.cc:186] Error getting master registration for 127.4.170.61:37949: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111),
I20260808 13:36:38.385144 11075 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:36:38.388782 11336 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" instance_seqno: 1786196192631127) as {username='slave'} at 127.4.170.3:51605; Asking this server to re-register.
I20260808 13:36:38.390955 10987 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:38.395533 11336 ts_manager.cc:194] Registered new tserver with Master: e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:38.398664 11436 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 026dcc55447e4441b062e327f64eacb4)
I20260808 13:36:38.399236 11436 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:38.401551 11436 leader_election.cc:288] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:38.422909 9805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111) [suppressed 4 similar messages]
W20260808 13:36:38.434190 9805 leader_election.cc:334] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949): Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:38.436049 11351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 5 candidate_status { last_received { term: 4 index: 31 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:38.437093 11351 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc2c297fb28f4950864f0487ef83e833 in term 3.
I20260808 13:36:38.438265 9809 leader_election.cc:302] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833; no voters: 026dcc55447e4441b062e327f64eacb4
I20260808 13:36:38.439429 11436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260808 13:36:38.439751 11436 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 026dcc55447e4441b062e327f64eacb4)
I20260808 13:36:38.440109 11436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 4 FOLLOWER]: Advancing to term 5
I20260808 13:36:38.449261 11436 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:38.453099 11351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc2c297fb28f4950864f0487ef83e833" candidate_term: 5 candidate_status { last_received { term: 4 index: 31 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534"
I20260808 13:36:38.453650 11351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 3 FOLLOWER]: Advancing to term 5
W20260808 13:36:38.455837 9805 leader_election.cc:334] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949): Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:38.458750 11436 leader_election.cc:288] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 election: Requested vote from peers 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:38.468623 11351 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc2c297fb28f4950864f0487ef83e833 in term 5.
I20260808 13:36:38.471814 9809 leader_election.cc:302] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833; no voters: 026dcc55447e4441b062e327f64eacb4
I20260808 13:36:38.473193 11436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 5 FOLLOWER]: Leader election won for term 5
I20260808 13:36:38.475288 11436 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [term 5 LEADER]: Becoming Leader. State: Replica: bc2c297fb28f4950864f0487ef83e833, State: Running, Role: LEADER
I20260808 13:36:38.476279 11436 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 31, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:38.481163 11440 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc2c297fb28f4950864f0487ef83e833. Latest consensus state: current_term: 5 leader_uuid: "bc2c297fb28f4950864f0487ef83e833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:38.482041 11440 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:38.484328 11442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:36:38.496030 11442 catalog_manager.cc:669] Loaded metadata for table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]
I20260808 13:36:38.498140 11442 catalog_manager.cc:669] Loaded metadata for table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]
I20260808 13:36:38.500438 11442 catalog_manager.cc:669] Loaded metadata for table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]
I20260808 13:36:38.508937 11442 tablet_loader.cc:96] loaded metadata for tablet 39172037294d4e369b3e54adca3f5c50 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14])
I20260808 13:36:38.512467 11442 tablet_loader.cc:96] loaded metadata for tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639])
I20260808 13:36:38.514994 11442 tablet_loader.cc:96] loaded metadata for tablet bdb23f24298544528289f064911e0d16 (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b])
I20260808 13:36:38.517421 11442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:36:38.534683 11442 catalog_manager.cc:1259] Loaded cluster ID: 9eb2bfe6721b4b6487ca35bc443651ed
I20260808 13:36:38.535091 11442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:36:38.545012 11442 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:36:38.551110 11442 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: Loaded TSK: 0
I20260808 13:36:38.553030 11442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:36:38.556157 10730 heartbeater.cc:347] Connected to a master server at 127.4.170.60:45415
I20260808 13:36:38.559053 11442 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260808 13:36:38.561728 11336 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" instance_seqno: 1786196190621954) as {username='slave'} at 127.4.170.2:58693; Asking this server to re-register.
I20260808 13:36:38.563665 10730 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:38.567013 11442 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260808 13:36:38.567636 11336 ts_manager.cc:194] Registered new tserver with Master: e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:38.599915 11351 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Refusing update from remote peer bc2c297fb28f4950864f0487ef83e833: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20260808 13:36:38.601738 11440 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 5, Time since last communication: 0.000s
W20260808 13:36:38.605144 9805 consensus_peers.cc:487] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 -> Peer 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949): Couldn't send request to peer 026dcc55447e4441b062e327f64eacb4. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 13:36:38.618757 9832 master_service.cc:946] ReplaceTablet: received request to replace tablet 39172037294d4e369b3e54adca3f5c50 from {username='slave'} at 127.0.0.1:49722
I20260808 13:36:38.636271 9819 master_service.cc:946] ReplaceTablet: received request to replace tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 from {username='slave'} at 127.0.0.1:49722
I20260808 13:36:38.620100 9829 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49722:
name: "default.Table_328786206e564026aa1c27ae85cf90f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:38.621472 9828 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49722:
table { table_name: "default.Table_575a74e193bc45fd99dcd06ae9318532" } modify_external_catalogs: true
I20260808 13:36:38.701460 9828 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49722:
table { table_name: "default.Table_575a74e193bc45fd99dcd06ae9318532" } modify_external_catalogs: true
I20260808 13:36:38.630738 9827 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49722:
table { table_name: "default.Table_f7b46abc59bd4cacacf793d1d5a184b6" } new_table_name: "default.Table_7a5955f4f27e416a846a78664eb3987a" modify_external_catalogs: true
I20260808 13:36:38.632851 9826 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49722:
name: "default.Table_face27da876543c6b6151d51b3eef9e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:38.634552 9822 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49722:
name: "default.Table_6a12fa2b84814dcbb515f51589cbf401"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:38.637151 9818 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49722:
table { table_name: "default.Table_82a5f186cbcd4cf0875234550bb2665e" } new_table_name: "default.Table_14527b1399a7419c90db6dbec907a64b" modify_external_catalogs: true
I20260808 13:36:38.607065 9831 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49722:
name: "default.Table_e071104baa0d4c3baca39e9ac21c484f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260808 13:36:38.721897 10269 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_82a5f186cbcd4cf0875234550bb2665e
*** Check failure stack trace: ***
I20260808 13:36:38.742654 11405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc2c297fb28f4950864f0487ef83e833. Latest consensus state: current_term: 5 leader_uuid: "bc2c297fb28f4950864f0487ef83e833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:38.743511 11405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:38.745952 11336 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "117d8c07fa254197969de761dd678aa6" instance_seqno: 1786196188527986) as {username='slave'} at 127.4.170.1:53705; Asking this server to re-register.
I20260808 13:36:38.747946 10492 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:38.748322 11451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "bc2c297fb28f4950864f0487ef83e833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:38.749055 11451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:38.753619 11336 ts_manager.cc:194] Registered new tserver with Master: 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471)
I20260808 13:36:38.759761 11451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "bc2c297fb28f4950864f0487ef83e833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:38.760574 11451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:38.775610 11436 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "bc2c297fb28f4950864f0487ef83e833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:38.776414 11436 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:38.791622 9832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: Sending DeleteTablet for 3 replicas of tablet 39172037294d4e369b3e54adca3f5c50
I20260808 13:36:38.800560 9832 catalog_manager.cc:6565] ReplaceTablet: tablet 39172037294d4e369b3e54adca3f5c50 deleted and replaced by tablet 9c766aac24e54c75a0e6202245a63234
I20260808 13:36:38.806713 9819 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: Sending DeleteTablet for 3 replicas of tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4
I20260808 13:36:38.808161 9819 catalog_manager.cc:6565] ReplaceTablet: tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 deleted and replaced by tablet 662cbf673bf94bde9026d19cb51b9193
I20260808 13:36:38.832584 9816 catalog_manager.cc:5495] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 117d8c07fa254197969de761dd678aa6 (127.4.170.1). New cstate: current_term: 1 leader_uuid: "117d8c07fa254197969de761dd678aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } health_report { overall_health: HEALTHY } } }
I20260808 13:36:38.874729 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet 39172037294d4e369b3e54adca3f5c50 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c766aac24e54c75a0e6202245a63234) from {username='slave'} at 127.0.0.1:41820
I20260808 13:36:38.879069 10496 heartbeater.cc:503] Master 127.4.170.62:46707 was elected leader, sending a full tablet report...
@ 0x7fb674eaa9a0 google::LogMessage::Flush() at ??:0
I20260808 13:36:38.887061 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 with delete_type TABLET_DATA_DELETED (replaced by tablet 662cbf673bf94bde9026d19cb51b9193) from {username='slave'} at 127.0.0.1:41820
I20260808 13:36:38.890702 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 9c766aac24e54c75a0e6202245a63234 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:38.891949 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c766aac24e54c75a0e6202245a63234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:38.901448 10868 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 with delete_type TABLET_DATA_DELETED (replaced by tablet 662cbf673bf94bde9026d19cb51b9193) from {username='slave'} at 127.0.0.1:49866
I20260808 13:36:38.901674 10867 tablet_service.cc:1514] Processing DeleteTablet for tablet 39172037294d4e369b3e54adca3f5c50 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c766aac24e54c75a0e6202245a63234) from {username='slave'} at 127.0.0.1:49866
I20260808 13:36:38.903451 10865 tablet_service.cc:1467] Processing CreateTablet for tablet 662cbf673bf94bde9026d19cb51b9193 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:38.904685 10865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 662cbf673bf94bde9026d19cb51b9193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:38.907055 10866 tablet_service.cc:1467] Processing CreateTablet for tablet 9c766aac24e54c75a0e6202245a63234 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:38.907542 10396 tablet_service.cc:1514] Processing DeleteTablet for tablet 39172037294d4e369b3e54adca3f5c50 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c766aac24e54c75a0e6202245a63234) from {username='slave'} at 127.0.0.1:40346
I20260808 13:36:38.908142 10866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c766aac24e54c75a0e6202245a63234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:38.915462 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 with delete_type TABLET_DATA_DELETED (replaced by tablet 662cbf673bf94bde9026d19cb51b9193) from {username='slave'} at 127.0.0.1:40346
I20260808 13:36:38.917834 10626 tablet_service.cc:1467] Processing CreateTablet for tablet 662cbf673bf94bde9026d19cb51b9193 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:38.919073 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 662cbf673bf94bde9026d19cb51b9193. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fb674eaa70f google::LogMessage::~LogMessage() at ??:0
I20260808 13:36:38.932001 10393 tablet_service.cc:1467] Processing CreateTablet for tablet 9c766aac24e54c75a0e6202245a63234 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:38.933386 10393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c766aac24e54c75a0e6202245a63234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:38.937804 11472 tablet_replica.cc:330] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:38.938695 11472 raft_consensus.cc:2229] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:36:38.942018 11449 mvcc.cc:205] Tried to move back new op lower bound from 7316259629431152640 to 7316259607304257536. Current Snapshot: MvccSnapshot[applied={T|T < 7316259629431152640}]
I20260808 13:36:38.944801 10396 tablet_service.cc:1467] Processing CreateTablet for tablet 662cbf673bf94bde9026d19cb51b9193 (DEFAULT_TABLE table=default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:38.945909 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 662cbf673bf94bde9026d19cb51b9193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:38.949920 11472 raft_consensus.cc:2258] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:38.952178 11472 ts_tablet_manager.cc:1905] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:38.962117 11471 tablet_replica.cc:330] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
I20260808 13:36:38.962998 11471 raft_consensus.cc:2229] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:38.963543 11471 raft_consensus.cc:2258] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:38.965555 11471 ts_tablet_manager.cc:1905] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:38.973635 11467 tablet_replica.cc:330] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:38.978062 11475 tablet_bootstrap.cc:492] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
@ 0x7fb67ff45aea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260808 13:36:38.974483 11467 raft_consensus.cc:2229] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:38.991374 11467 raft_consensus.cc:2258] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:38.993438 11467 ts_tablet_manager.cc:1905] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:39.008965 9819 master_service.cc:946] ReplaceTablet: received request to replace tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 from {username='slave'} at 127.0.0.1:49722
I20260808 13:36:39.015069 9816 master_service.cc:946] ReplaceTablet: received request to replace tablet bdb23f24298544528289f064911e0d16 from {username='slave'} at 127.0.0.1:49722
I20260808 13:36:39.015884 11475 tablet_bootstrap.cc:654] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:39.044495 9816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: Sending DeleteTablet for 3 replicas of tablet bdb23f24298544528289f064911e0d16
I20260808 13:36:39.046474 11475 tablet_bootstrap.cc:492] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:39.046916 11475 ts_tablet_manager.cc:1397] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.069s user 0.002s sys 0.010s
I20260808 13:36:39.048588 10866 tablet_service.cc:1514] Processing DeleteTablet for tablet bdb23f24298544528289f064911e0d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 9494b956367643ca9f4372a94bd84cc5) from {username='slave'} at 127.0.0.1:49866
I20260808 13:36:39.049518 11475 raft_consensus.cc:348] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.050410 11475 raft_consensus.cc:374] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:39.050743 11475 raft_consensus.cc:729] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:39.038522 11472 ts_tablet_manager.cc:1918] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:39.051631 11485 tablet_bootstrap.cc:492] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:39.051973 11472 log.cc:1196] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/39172037294d4e369b3e54adca3f5c50
I20260808 13:36:39.053192 11472 ts_tablet_manager.cc:1939] T 39172037294d4e369b3e54adca3f5c50 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:39.054764 11475 consensus_queue.cc:260] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.056026 11471 ts_tablet_manager.cc:1918] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:39.056433 11471 log.cc:1196] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/6b2ec07c8108414f8a27a5a3c3c1cbc4
I20260808 13:36:39.057192 11471 ts_tablet_manager.cc:1939] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:39.057410 9816 catalog_manager.cc:6565] ReplaceTablet: tablet bdb23f24298544528289f064911e0d16 deleted and replaced by tablet 9494b956367643ca9f4372a94bd84cc5
I20260808 13:36:39.057993 11485 tablet_bootstrap.cc:654] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:39.058195 10393 tablet_service.cc:1514] Processing DeleteTablet for tablet bdb23f24298544528289f064911e0d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 9494b956367643ca9f4372a94bd84cc5) from {username='slave'} at 127.0.0.1:40346
I20260808 13:36:39.062783 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet bdb23f24298544528289f064911e0d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 9494b956367643ca9f4372a94bd84cc5) from {username='slave'} at 127.0.0.1:41820
I20260808 13:36:39.064204 11482 tablet_bootstrap.cc:492] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:39.071759 9811 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet 39172037294d4e369b3e54adca3f5c50 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:39.074083 9811 catalog_manager.cc:4841] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]) successfully deleted
I20260808 13:36:39.080377 10866 tablet_service.cc:1467] Processing CreateTablet for tablet 9494b956367643ca9f4372a94bd84cc5 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:39.080770 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 9494b956367643ca9f4372a94bd84cc5 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:39.081636 10866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9494b956367643ca9f4372a94bd84cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:39.081936 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9494b956367643ca9f4372a94bd84cc5. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fb67ff4533c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260808 13:36:39.074987 11472 tablet_replica.cc:330] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:39.091112 11485 tablet_bootstrap.cc:492] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:39.094321 11485 ts_tablet_manager.cc:1397] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.006s
I20260808 13:36:39.095073 11471 tablet_replica.cc:330] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
I20260808 13:36:39.095937 11472 raft_consensus.cc:2229] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:36:39.096917 11472 raft_consensus.cc:2258] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:39.096974 11485 raft_consensus.cc:348] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.097730 11485 raft_consensus.cc:374] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:39.097994 11485 raft_consensus.cc:729] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:39.087064 10396 tablet_service.cc:1467] Processing CreateTablet for tablet 9494b956367643ca9f4372a94bd84cc5 (DEFAULT_TABLE table=default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:39.099678 10396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9494b956367643ca9f4372a94bd84cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:39.095851 11482 tablet_bootstrap.cc:654] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:39.106067 11485 consensus_queue.cc:260] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.108341 11485 ts_tablet_manager.cc:1428] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260808 13:36:39.109177 11485 tablet_bootstrap.cc:492] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
@ 0x7fb67ff48fd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260808 13:36:39.124667 11475 ts_tablet_manager.cc:1428] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.077s user 0.007s sys 0.000s
I20260808 13:36:39.131067 10991 heartbeater.cc:503] Master 127.4.170.62:46707 was elected leader, sending a full tablet report...
W20260808 13:36:39.138056 9816 catalog_manager.cc:5328] Ignoring report from unknown tablet 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:36:39.138726 9816 catalog_manager.cc:5328] Ignoring report from unknown tablet 39172037294d4e369b3e54adca3f5c50
W20260808 13:36:39.138954 9816 catalog_manager.cc:5328] Ignoring report from unknown tablet 7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:39.143044 11485 tablet_bootstrap.cc:654] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:39.143378 11475 tablet_bootstrap.cc:492] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
@ 0x7fb67ff48f8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260808 13:36:39.149636 11471 raft_consensus.cc:2229] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:39.150221 11471 raft_consensus.cc:2258] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:39.153380 11471 ts_tablet_manager.cc:1905] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:39.153828 11482 tablet_bootstrap.cc:492] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:39.154238 11482 ts_tablet_manager.cc:1397] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.090s user 0.012s sys 0.000s
I20260808 13:36:39.160323 11482 raft_consensus.cc:348] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.162850 11482 raft_consensus.cc:374] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:39.163120 11482 raft_consensus.cc:729] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:39.163771 11482 consensus_queue.cc:260] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.167037 11485 tablet_bootstrap.cc:492] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:39.167459 11485 ts_tablet_manager.cc:1397] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.058s user 0.010s sys 0.000s
I20260808 13:36:39.169945 11485 raft_consensus.cc:348] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.170643 11485 raft_consensus.cc:374] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:39.170910 11485 raft_consensus.cc:729] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:39.171654 11485 consensus_queue.cc:260] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.178293 11482 ts_tablet_manager.cc:1428] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.024s user 0.004s sys 0.004s
@ 0x7fb67ff48f1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260808 13:36:39.185979 11485 ts_tablet_manager.cc:1428] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260808 13:36:39.189199 11485 tablet_bootstrap.cc:492] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:39.202711 11472 ts_tablet_manager.cc:1905] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fb67ff48ee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260808 13:36:39.207350 11485 tablet_bootstrap.cc:654] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:39.210533 11475 tablet_bootstrap.cc:654] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:39.231715 11485 tablet_bootstrap.cc:492] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:39.232141 11485 ts_tablet_manager.cc:1397] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.043s user 0.009s sys 0.000s
I20260808 13:36:39.232779 9816 master_service.cc:946] ReplaceTablet: received request to replace tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 from {username='slave'} at 127.0.0.1:49722
I20260808 13:36:39.234674 11485 raft_consensus.cc:348] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.235339 11485 raft_consensus.cc:374] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:39.235586 11485 raft_consensus.cc:729] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:39.242944 11485 consensus_queue.cc:260] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.248653 11498 raft_consensus.cc:482] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:39.252923 11485 ts_tablet_manager.cc:1428] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260808 13:36:39.249233 11498 raft_consensus.cc:504] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.258745 11498 leader_election.cc:288] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:39.261703 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c766aac24e54c75a0e6202245a63234" candidate_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d8c07fa254197969de761dd678aa6" is_pre_election: true
I20260808 13:36:39.262446 10423 raft_consensus.cc:2454] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0e40695b6c9401aa0c178f58d8e2f37 in term 0.
I20260808 13:36:39.264297 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c766aac24e54c75a0e6202245a63234" candidate_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995" is_pre_election: true
I20260808 13:36:39.265267 10657 raft_consensus.cc:2454] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0e40695b6c9401aa0c178f58d8e2f37 in term 0.
I20260808 13:36:39.266637 10791 leader_election.cc:302] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:39.268234 11273 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:39.268667 11273 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.269861 11498 raft_consensus.cc:2781] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:39.270167 11498 raft_consensus.cc:482] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:39.270470 11498 raft_consensus.cc:3037] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:39.270596 11273 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:39.273226 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9494b956367643ca9f4372a94bd84cc5" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" is_pre_election: true
I20260808 13:36:39.273666 11273 raft_consensus.cc:482] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:39.274044 11273 raft_consensus.cc:504] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
W20260808 13:36:39.274828 10315 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:39.276880 11472 ts_tablet_manager.cc:1918] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:39.276793 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662cbf673bf94bde9026d19cb51b9193" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995" is_pre_election: true
I20260808 13:36:39.277249 11472 log.cc:1196] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/6b2ec07c8108414f8a27a5a3c3c1cbc4
I20260808 13:36:39.278003 11472 ts_tablet_manager.cc:1939] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
W20260808 13:36:39.278371 10313 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:36:39.278795 10313 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117d8c07fa254197969de761dd678aa6; no voters: e0e40695b6c9401aa0c178f58d8e2f37, e9d3df921f204d8aa02da52aca33d995
I20260808 13:36:39.279629 11273 leader_election.cc:288] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:39.280063 11475 tablet_bootstrap.cc:492] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:39.280092 11273 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:39.280187 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662cbf673bf94bde9026d19cb51b9193" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" is_pre_election: true
I20260808 13:36:39.280495 11475 ts_tablet_manager.cc:1397] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.137s user 0.018s sys 0.001s
I20260808 13:36:39.280690 9811 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]) successfully deleted
@ 0x7fb67ff481de std::__1::__function::__func<>::operator()() at ??:0
I20260808 13:36:39.284132 9816 master_service.cc:946] ReplaceTablet: received request to replace tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 from {username='slave'} at 127.0.0.1:49722
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
W20260808 13:36:39.286279 10315 leader_election.cc:341] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:39.287710 11472 tablet_replica.cc:330] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:39.288451 11472 raft_consensus.cc:2229] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:36:39.289289 11472 raft_consensus.cc:2258] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:36:39.291368 10313 leader_election.cc:341] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:39.291750 10313 leader_election.cc:302] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117d8c07fa254197969de761dd678aa6; no voters: e0e40695b6c9401aa0c178f58d8e2f37, e9d3df921f204d8aa02da52aca33d995
@ 0x5805ba std::__1::function<>::operator()() at ??:?
I20260808 13:36:39.294473 11507 raft_consensus.cc:2726] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:39.295421 10732 heartbeater.cc:503] Master 127.4.170.62:46707 was elected leader, sending a full tablet report...
I20260808 13:36:39.077436 11467 ts_tablet_manager.cc:1918] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:39.297453 11467 log.cc:1196] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/39172037294d4e369b3e54adca3f5c50
I20260808 13:36:39.298646 11467 ts_tablet_manager.cc:1939] T 39172037294d4e369b3e54adca3f5c50 P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:39.301019 11482 tablet_bootstrap.cc:492] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
W20260808 13:36:39.305632 9816 catalog_manager.cc:5328] Ignoring report from unknown tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4
W20260808 13:36:39.306332 9816 catalog_manager.cc:5328] Ignoring report from unknown tablet 39172037294d4e369b3e54adca3f5c50
W20260808 13:36:39.306553 9816 catalog_manager.cc:5328] Ignoring report from unknown tablet 7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:39.309749 11498 raft_consensus.cc:504] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.314724 9807 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet 39172037294d4e369b3e54adca3f5c50 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:39.315793 11467 tablet_replica.cc:330] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:39.317320 11467 raft_consensus.cc:2229] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:39.317852 11467 raft_consensus.cc:2258] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:39.293859 11475 raft_consensus.cc:348] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.319698 11472 ts_tablet_manager.cc:1905] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:39.322464 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c766aac24e54c75a0e6202245a63234" candidate_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d8c07fa254197969de761dd678aa6"
I20260808 13:36:39.323061 10423 raft_consensus.cc:3037] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:39.323173 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c766aac24e54c75a0e6202245a63234" candidate_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995"
I20260808 13:36:39.323727 10657 raft_consensus.cc:3037] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:39.329939 10423 raft_consensus.cc:2454] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e40695b6c9401aa0c178f58d8e2f37 in term 1.
I20260808 13:36:39.330994 10796 leader_election.cc:302] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e0e40695b6c9401aa0c178f58d8e2f37; no voters:
I20260808 13:36:39.331192 10657 raft_consensus.cc:2454] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e40695b6c9401aa0c178f58d8e2f37 in term 1.
I20260808 13:36:39.332057 11486 raft_consensus.cc:2781] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:39.332957 11482 tablet_bootstrap.cc:654] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:39.334913 11498 leader_election.cc:288] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 election: Requested vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:39.340332 11467 ts_tablet_manager.cc:1905] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:39.320541 11475 raft_consensus.cc:374] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:39.346982 11475 raft_consensus.cc:729] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:39.355032 11475 consensus_queue.cc:260] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.361338 11486 raft_consensus.cc:686] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 LEADER]: Becoming Leader. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Running, Role: LEADER
I20260808 13:36:39.362275 11486 consensus_queue.cc:237] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.369740 11482 tablet_bootstrap.cc:492] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:39.373679 11471 ts_tablet_manager.cc:1918] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:39.374118 11471 log.cc:1196] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/39172037294d4e369b3e54adca3f5c50
I20260808 13:36:39.374930 11471 ts_tablet_manager.cc:1939] T 39172037294d4e369b3e54adca3f5c50 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:39.378417 9811 catalog_manager.cc:4841] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): tablet 39172037294d4e369b3e54adca3f5c50 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:39.380065 11475 ts_tablet_manager.cc:1428] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.099s user 0.006s sys 0.002s
I20260808 13:36:39.381496 11475 tablet_bootstrap.cc:492] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:39.388725 11511 raft_consensus.cc:482] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:39.370236 11482 ts_tablet_manager.cc:1397] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.070s user 0.010s sys 0.018s
I20260808 13:36:39.389211 11511 raft_consensus.cc:504] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.392508 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662cbf673bf94bde9026d19cb51b9193" candidate_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d8c07fa254197969de761dd678aa6" is_pre_election: true
I20260808 13:36:39.393169 10423 raft_consensus.cc:2454] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0e40695b6c9401aa0c178f58d8e2f37 in term 0.
I20260808 13:36:39.393203 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662cbf673bf94bde9026d19cb51b9193" candidate_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995" is_pre_election: true
I20260808 13:36:39.393850 11482 raft_consensus.cc:348] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.394646 11511 leader_election.cc:288] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:39.394521 10796 leader_election.cc:302] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e0e40695b6c9401aa0c178f58d8e2f37; no voters:
W20260808 13:36:39.395844 10791 leader_election.cc:341] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Illegal state: must be running to vote when last-logged opid is not known
I20260808 13:36:39.394774 11482 raft_consensus.cc:374] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:39.399104 11482 raft_consensus.cc:729] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:39.400005 11482 consensus_queue.cc:260] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.403415 11472 ts_tablet_manager.cc:1918] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:39.403793 11472 log.cc:1196] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/bdb23f24298544528289f064911e0d16
I20260808 13:36:39.404547 11472 ts_tablet_manager.cc:1939] T bdb23f24298544528289f064911e0d16 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:39.401109 11511 raft_consensus.cc:2781] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:39.409873 11482 ts_tablet_manager.cc:1428] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.020s user 0.005s sys 0.001s
I20260808 13:36:39.410076 11511 raft_consensus.cc:482] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:39.410410 11511 raft_consensus.cc:3037] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:39.411077 11471 tablet_replica.cc:330] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
I20260808 13:36:39.411859 11471 raft_consensus.cc:2229] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:39.412349 11471 raft_consensus.cc:2258] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:39.418028 11471 ts_tablet_manager.cc:1905] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:39.421805 11482 tablet_bootstrap.cc:492] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:39.416226 11475 tablet_bootstrap.cc:654] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:39.435441 9811 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet bdb23f24298544528289f064911e0d16 (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
I20260808 13:36:39.440434 11467 ts_tablet_manager.cc:1918] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:39.440825 11467 log.cc:1196] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/6b2ec07c8108414f8a27a5a3c3c1cbc4
I20260808 13:36:39.441116 9819 master_service.cc:946] ReplaceTablet: received request to replace tablet 9494b956367643ca9f4372a94bd84cc5 from {username='slave'} at 127.0.0.1:49722
I20260808 13:36:39.432385 11482 tablet_bootstrap.cc:654] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:39.446724 11467 ts_tablet_manager.cc:1939] T 6b2ec07c8108414f8a27a5a3c3c1cbc4 P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:39.453114 11467 tablet_replica.cc:330] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:39.453902 11467 raft_consensus.cc:2229] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:39.454464 11467 raft_consensus.cc:2258] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:39.456518 11467 ts_tablet_manager.cc:1905] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:39.461557 9807 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet 6b2ec07c8108414f8a27a5a3c3c1cbc4 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]) successfully deleted
I20260808 13:36:39.469471 9819 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc2c297fb28f4950864f0487ef83e833: Sending DeleteTablet for 3 replicas of tablet 9494b956367643ca9f4372a94bd84cc5
I20260808 13:36:39.471305 10396 tablet_service.cc:1514] Processing DeleteTablet for tablet 9494b956367643ca9f4372a94bd84cc5 with delete_type TABLET_DATA_DELETED (replaced by tablet 5774a0e0e00e469695ba21c2b051390c) from {username='slave'} at 127.0.0.1:40346
I20260808 13:36:39.472079 11472 tablet_replica.cc:330] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:39.472649 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 9494b956367643ca9f4372a94bd84cc5 with delete_type TABLET_DATA_DELETED (replaced by tablet 5774a0e0e00e469695ba21c2b051390c) from {username='slave'} at 127.0.0.1:41820
I20260808 13:36:39.472841 11472 raft_consensus.cc:2229] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:39.473299 11472 raft_consensus.cc:2258] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:39.475909 9819 catalog_manager.cc:6565] ReplaceTablet: tablet 9494b956367643ca9f4372a94bd84cc5 deleted and replaced by tablet 5774a0e0e00e469695ba21c2b051390c
I20260808 13:36:39.476195 11472 ts_tablet_manager.cc:1905] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:39.487877 11482 tablet_bootstrap.cc:492] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:39.488313 11482 ts_tablet_manager.cc:1397] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.067s user 0.014s sys 0.012s
I20260808 13:36:39.491394 11482 raft_consensus.cc:348] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.492079 11482 raft_consensus.cc:374] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:39.492362 11482 raft_consensus.cc:729] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:39.493124 11482 consensus_queue.cc:260] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.500629 11482 ts_tablet_manager.cc:1428] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260808 13:36:39.501763 11467 ts_tablet_manager.cc:1918] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:39.502140 11467 log.cc:1196] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/bdb23f24298544528289f064911e0d16
@ 0x7fb67705bdc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260808 13:36:39.502933 11467 ts_tablet_manager.cc:1939] T bdb23f24298544528289f064911e0d16 P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:39.505662 11472 ts_tablet_manager.cc:1918] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:39.506011 11472 log.cc:1196] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/9494b956367643ca9f4372a94bd84cc5
I20260808 13:36:39.506688 11472 ts_tablet_manager.cc:1939] T 9494b956367643ca9f4372a94bd84cc5 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:39.512600 9811 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet 9494b956367643ca9f4372a94bd84cc5 (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
I20260808 13:36:39.519150 11467 tablet_replica.cc:330] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
I20260808 13:36:39.519857 11467 raft_consensus.cc:2229] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:39.520277 11467 raft_consensus.cc:2258] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:39.520323 9807 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet bdb23f24298544528289f064911e0d16 (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]) successfully deleted
@ 0x7fb6786c96db start_thread at ??:0
I20260808 13:36:39.560318 9819 master_service.cc:946] ReplaceTablet: received request to replace tablet 9494b956367643ca9f4372a94bd84cc5 from {username='slave'} at 127.0.0.1:49722
I20260808 13:36:39.563220 11511 raft_consensus.cc:504] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.566553 10657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662cbf673bf94bde9026d19cb51b9193" candidate_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995"
I20260808 13:36:39.567130 10657 raft_consensus.cc:3037] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:39.567175 11511 leader_election.cc:288] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 election: Requested vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
@ 0x7fb673b4f61f clone at ??:0
I20260808 13:36:39.570808 11467 ts_tablet_manager.cc:1905] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:39.576136 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662cbf673bf94bde9026d19cb51b9193" candidate_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d8c07fa254197969de761dd678aa6"
I20260808 13:36:39.576701 10423 raft_consensus.cc:3037] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:39.578287 10866 tablet_service.cc:1514] Processing DeleteTablet for tablet 9494b956367643ca9f4372a94bd84cc5 with delete_type TABLET_DATA_DELETED (replaced by tablet 5774a0e0e00e469695ba21c2b051390c) from {username='slave'} at 127.0.0.1:49866
I20260808 13:36:39.581743 10657 raft_consensus.cc:2454] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e40695b6c9401aa0c178f58d8e2f37 in term 1.
I20260808 13:36:39.583096 10423 raft_consensus.cc:2454] T 662cbf673bf94bde9026d19cb51b9193 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e40695b6c9401aa0c178f58d8e2f37 in term 1.
I20260808 13:36:39.583025 10791 leader_election.cc:302] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:39.586162 11486 raft_consensus.cc:2781] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:39.586740 11486 raft_consensus.cc:686] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 LEADER]: Becoming Leader. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Running, Role: LEADER
I20260808 13:36:39.587697 11486 consensus_queue.cc:237] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.588171 11471 ts_tablet_manager.cc:1918] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:39.589393 11471 log.cc:1196] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/bdb23f24298544528289f064911e0d16
I20260808 13:36:39.590190 11471 ts_tablet_manager.cc:1939] T bdb23f24298544528289f064911e0d16 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:39.591320 11475 tablet_bootstrap.cc:492] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:39.613481 11475 ts_tablet_manager.cc:1397] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.232s user 0.023s sys 0.022s
I20260808 13:36:39.633535 11467 ts_tablet_manager.cc:1918] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260808 13:36:39.633170 11475 raft_consensus.cc:348] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.633900 11475 raft_consensus.cc:374] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:39.633949 11467 log.cc:1196] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/9494b956367643ca9f4372a94bd84cc5
I20260808 13:36:39.634171 11475 raft_consensus.cc:729] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:39.634590 11467 ts_tablet_manager.cc:1939] T 9494b956367643ca9f4372a94bd84cc5 P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:39.636556 11475 consensus_queue.cc:260] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:39.642949 11475 ts_tablet_manager.cc:1428] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
W20260808 13:36:39.923971 10796 connection.cc:535] server connection from 127.0.0.1:49866 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:39.924574 10796 connection.cc:535] client connection to 127.4.170.62:46707 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:39.925436 10560 connection.cc:535] server connection from 127.0.0.1:41820 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:39.926052 10315 connection.cc:535] client connection to 127.4.170.62:46707 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:39.926868 11010 connection.cc:535] client connection to 127.4.170.62:46707 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:39.928702 11010 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.4.170.62:46707)
W20260808 13:36:39.929467 11010 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.170.62:46707)
W20260808 13:36:39.930166 11010 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.4.170.62:46707)
W20260808 13:36:39.940076 11010 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.4.170.62:46707)
I20260808 13:36:40.052873 11542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bc2c297fb28f4950864f0487ef83e833)
I20260808 13:36:40.053330 11542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
W20260808 13:36:40.058872 11314 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:36:40.063570 11542 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949)
W20260808 13:36:40.080399 11314 leader_election.cc:334] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:36:40.081197 11310 leader_election.cc:334] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949): Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:40.081616 11310 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bf9978620f543d8aee6279b9a9d3534; no voters: 026dcc55447e4441b062e327f64eacb4, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:40.082417 11542 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260808 13:36:40.282150 11511 consensus_queue.cc:1035] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:40.310124 11557 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:40.312757 11554 consensus_queue.cc:1035] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:36:40.310875 11557 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
W20260808 13:36:40.318262 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:36:40.324841 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:36:40.325349 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:36:40.326411 11557 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:40.327380 11557 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:40.658872 11558 consensus_queue.cc:1035] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:40.691598 11554 consensus_queue.cc:1035] T 662cbf673bf94bde9026d19cb51b9193 P e0e40695b6c9401aa0c178f58d8e2f37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:36:40.775813 11573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bc2c297fb28f4950864f0487ef83e833)
I20260808 13:36:40.776393 11573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:40.778493 11573 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949)
W20260808 13:36:40.785606 11314 leader_election.cc:334] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:36:40.789342 11310 leader_election.cc:334] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949): Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:40.789794 11310 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bf9978620f543d8aee6279b9a9d3534; no voters: 026dcc55447e4441b062e327f64eacb4, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:40.790475 11573 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260808 13:36:40.793821 11558 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:40.794291 11558 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:40.796267 11558 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:36:40.798463 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:36:40.799065 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:36:40.799464 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:36:40.800197 11558 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:41.258406 4776 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
/tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.61:37949
--webserver_interface=127.4.170.61
--webserver_port=42377
--builtin_ntp_servers=127.4.170.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--hive_metastore_uris=thrift://127.0.0.1:46603
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260808 13:36:41.308810 11584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bc2c297fb28f4950864f0487ef83e833)
I20260808 13:36:41.309226 11584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:41.310933 11584 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949)
W20260808 13:36:41.317405 11314 leader_election.cc:334] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:36:41.317924 11310 leader_election.cc:334] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949): Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:41.318290 11310 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bf9978620f543d8aee6279b9a9d3534; no voters: 026dcc55447e4441b062e327f64eacb4, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:41.318918 11584 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260808 13:36:41.633708 11583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:36:41.634351 11583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:36:41.634972 11583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:36:41.669412 11583 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260808 13:36:41.669772 11583 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260808 13:36:41.670109 11583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:36:41.670444 11583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:36:41.670817 11583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:36:41.671151 11583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:36:41.714148 11583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.170.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:46603
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.170.61:37949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.170.61
--webserver_port=42377
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:50:05 UTC on ccycloud.quasar-ayrwuc.root.comops.site
build id 4574796
TSAN enabled
I20260808 13:36:41.715452 11583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:36:41.717052 11583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:36:41.739866 11601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:42.993465 11630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bc2c297fb28f4950864f0487ef83e833)
I20260808 13:36:42.994012 11630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:42.995783 11630 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949)
W20260808 13:36:43.014806 11314 leader_election.cc:334] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:36:43.015383 11310 leader_election.cc:334] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949): Network error: Client connection negotiation failed: client connection to 127.4.170.61:37949: connect: Connection refused (error 111)
I20260808 13:36:43.015818 11310 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bf9978620f543d8aee6279b9a9d3534; no voters: 026dcc55447e4441b062e327f64eacb4, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:43.016448 11630 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260808 13:36:41.739885 11605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:36:41.739893 11607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:36:41.742410 11583 server_base.cc:1034] running on GCE node
I20260808 13:36:43.071244 11583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:36:43.074538 11583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:36:43.075953 11583 hybrid_clock.cc:648] HybridClock initialized: now 1786196203075910 us; error 45 us; skew 500 ppm
I20260808 13:36:43.076838 11583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:36:43.092437 11583 webserver.cc:460] Webserver started at http://127.4.170.61:42377/ using document root <none> and password file <none>
I20260808 13:36:43.093544 11583 fs_manager.cc:359] Metadata directory not provided
I20260808 13:36:43.093770 11583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:36:43.103140 11583 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.006s sys 0.001s
I20260808 13:36:43.108487 11642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:36:43.109668 11583 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.005s
I20260808 13:36:43.110025 11583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data,/tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal
uuid: "026dcc55447e4441b062e327f64eacb4"
format_stamp: "Formatted at 2026-08-08 13:36:22 on dist-test-slave-twwt"
I20260808 13:36:43.112669 11583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:36:43.157567 11583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:36:43.159951 11583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:36:43.160557 11583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:36:43.266716 11583 rpc_server.cc:307] RPC server started. Bound to: 127.4.170.61:37949
I20260808 13:36:43.266903 11726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.170.61:37949 every 8 connection(s)
I20260808 13:36:43.269694 11583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/master-1/data/info.pb
I20260808 13:36:43.280143 4776 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoVZaNS/build/tsan/bin/kudu as pid 11583
I20260808 13:36:43.281301 11727 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:36:43.288780 11727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Bootstrap starting.
I20260808 13:36:43.323375 11727 log.cc:824] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Log is configured to *not* fsync() on all Append() calls
I20260808 13:36:43.387568 11075 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:36:43.387933 11075 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:36:43.410252 11727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=18 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:36:43.411177 11727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: Bootstrap complete.
I20260808 13:36:43.431154 11727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:43.433560 11727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 026dcc55447e4441b062e327f64eacb4, State: Initialized, Role: FOLLOWER
I20260808 13:36:43.434463 11727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 4.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:43.437028 11735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:43.437700 11735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:43.438297 11727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:36:43.475345 11757 catalog_manager.cc:1259] Loaded cluster ID: 9eb2bfe6721b4b6487ca35bc443651ed
I20260808 13:36:43.475659 11757 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: loading cluster ID for follower catalog manager: success
I20260808 13:36:43.490995 11757 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: acquiring CA information for follower catalog manager: success
I20260808 13:36:43.498665 11757 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:36:43.481006 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:36:43.506114 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:36:43.710268 10494 heartbeater.cc:347] Connected to a master server at 127.4.170.61:37949
I20260808 13:36:43.717387 11665 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "117d8c07fa254197969de761dd678aa6" instance_seqno: 1786196188527986) as {username='slave'} at 127.4.170.1:38531; Asking this server to re-register.
I20260808 13:36:43.718976 10494 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:43.722019 11665 ts_manager.cc:194] Registered new tserver with Master: 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471)
I20260808 13:36:43.745446 11685 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:43.746145 11685 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:43.748438 11685 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:43.754405 10989 heartbeater.cc:347] Connected to a master server at 127.4.170.61:37949
W20260808 13:36:43.754724 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:36:43.761930 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:36:43.762425 10796 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:36:43.763782 11761 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:43.764873 10731 heartbeater.cc:347] Connected to a master server at 127.4.170.61:37949
I20260808 13:36:43.765177 11665 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" instance_seqno: 1786196192631127) as {username='slave'} at 127.4.170.3:36783; Asking this server to re-register.
I20260808 13:36:43.766901 10989 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:43.769311 11665 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" instance_seqno: 1786196190621954) as {username='slave'} at 127.4.170.2:55607; Asking this server to re-register.
I20260808 13:36:43.771292 11664 ts_manager.cc:194] Registered new tserver with Master: e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:43.772625 10731 heartbeater.cc:464] Registering TS with master...
I20260808 13:36:43.775621 11664 ts_manager.cc:194] Registered new tserver with Master: e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:36:43.824389 11735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:43.824975 11735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
W20260808 13:36:43.833140 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:36:43.833204 11735 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:43.846630 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:36:43.850265 11349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 5 candidate_status { last_received { term: 4 index: 31 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:43.851205 11349 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 026dcc55447e4441b062e327f64eacb4 in current term 5: Already voted for candidate bc2c297fb28f4950864f0487ef83e833 in this term.
I20260808 13:36:43.852515 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:43.853351 11735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 4 FOLLOWER]: Advancing to term 5
I20260808 13:36:43.862813 11735 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260808 13:36:44.446929 11780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:44.447393 11780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:44.449477 11780 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:44.450778 11349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 6 candidate_status { last_received { term: 4 index: 31 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:44.451910 11349 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 026dcc55447e4441b062e327f64eacb4 for term 6 because replica has last-logged OpId of term: 5 index: 43, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 31.
W20260808 13:36:44.454941 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:36:44.455353 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:44.456063 11780 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260808 13:36:45.095717 11791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:45.096119 11791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:45.097944 11791 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
I20260808 13:36:45.099752 11349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "026dcc55447e4441b062e327f64eacb4" candidate_term: 6 candidate_status { last_received { term: 4 index: 31 } } ignore_live_leader: false dest_uuid: "0bf9978620f543d8aee6279b9a9d3534" is_pre_election: true
I20260808 13:36:45.101217 11349 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 026dcc55447e4441b062e327f64eacb4 for term 6 because replica has last-logged OpId of term: 5 index: 43, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 31.
W20260808 13:36:45.105176 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:36:45.105590 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:45.106292 11791 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260808 13:36:45.120108 11795 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:45.120579 11795 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:45.122754 11795 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:36:45.124614 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:36:45.125249 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:36:45.125622 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:36:45.135347 11795 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:45.369450 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 6: this message will repeat every 5th retry.
I20260808 13:36:45.470742 11802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bc2c297fb28f4950864f0487ef83e833)
I20260808 13:36:45.471251 11802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:45.473277 11802 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949)
W20260808 13:36:45.475999 11314 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20260808 13:36:45.479522 11314 leader_election.cc:334] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:36:45.486970 11680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 6 candidate_status { last_received { term: 5 index: 43 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4" is_pre_election: true
I20260808 13:36:45.487677 11680 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf9978620f543d8aee6279b9a9d3534 in term 5.
I20260808 13:36:45.488855 11310 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534; no voters: bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:45.489589 11802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260808 13:36:45.489969 11802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Starting leader election (detected failure of leader bc2c297fb28f4950864f0487ef83e833)
I20260808 13:36:45.490255 11802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 5 FOLLOWER]: Advancing to term 6
I20260808 13:36:45.496052 11802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:45.497841 11802 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 election: Requested vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 026dcc55447e4441b062e327f64eacb4 (127.4.170.61:37949)
I20260808 13:36:45.498997 11680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bf9978620f543d8aee6279b9a9d3534" candidate_term: 6 candidate_status { last_received { term: 5 index: 43 } } ignore_live_leader: false dest_uuid: "026dcc55447e4441b062e327f64eacb4"
I20260808 13:36:45.499567 11680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 5 FOLLOWER]: Advancing to term 6
W20260808 13:36:45.500942 11314 leader_election.cc:334] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:36:45.504792 11680 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf9978620f543d8aee6279b9a9d3534 in term 6.
I20260808 13:36:45.505654 11310 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4, 0bf9978620f543d8aee6279b9a9d3534; no voters: bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:45.506238 11802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 6 FOLLOWER]: Leader election won for term 6
I20260808 13:36:45.507500 11802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [term 6 LEADER]: Becoming Leader. State: Replica: 0bf9978620f543d8aee6279b9a9d3534, State: Running, Role: LEADER
I20260808 13:36:45.508337 11802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 5.43, Last appended by leader: 43, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:45.512737 11810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf9978620f543d8aee6279b9a9d3534. Latest consensus state: current_term: 6 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:45.513705 11810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: LEADER
I20260808 13:36:45.516062 11812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:36:45.524499 11812 catalog_manager.cc:669] Loaded metadata for table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b]
I20260808 13:36:45.525810 11812 catalog_manager.cc:669] Loaded metadata for table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]
I20260808 13:36:45.527148 11812 catalog_manager.cc:669] Loaded metadata for table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639]
I20260808 13:36:45.533044 11812 tablet_loader.cc:96] loaded metadata for tablet 5774a0e0e00e469695ba21c2b051390c (table default.table_82a5f186cbcd4cf0875234550bb2665e [id=1c41141d42bb4166b8a3a461f504c61b])
I20260808 13:36:45.534256 11812 tablet_loader.cc:96] loaded metadata for tablet 662cbf673bf94bde9026d19cb51b9193 (table default.table_f7b46abc59bd4cacacf793d1d5a184b6 [id=d9ec6ad6315a4df99b9739a8b5b5f639])
I20260808 13:36:45.536046 11812 tablet_loader.cc:96] loaded metadata for tablet 9c766aac24e54c75a0e6202245a63234 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14])
I20260808 13:36:45.537649 11812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:36:45.541380 11812 catalog_manager.cc:1259] Loaded cluster ID: 9eb2bfe6721b4b6487ca35bc443651ed
I20260808 13:36:45.541548 11812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:36:45.546370 11812 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:36:45.550244 11812 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Loaded TSK: 0
I20260808 13:36:45.551472 11812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:36:45.555215 11812 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260808 13:36:45.558537 11812 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260808 13:36:45.573587 11680 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Refusing update from remote peer 0bf9978620f543d8aee6279b9a9d3534: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 6 index: 45. (index mismatch)
I20260808 13:36:45.575410 11810 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [LEADER]: Connected to new peer: Peer: permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 31, Time since last communication: 0.001s
W20260808 13:36:45.576256 11314 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 -> Peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Couldn't send request to peer bc2c297fb28f4950864f0487ef83e833. Status: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 13:36:45.594328 11337 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:40570:
table { table_name: "default.Table_82a5f186cbcd4cf0875234550bb2665e" } new_table_name: "default.Table_14527b1399a7419c90db6dbec907a64b" modify_external_catalogs: true
I20260808 13:36:45.595737 11336 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40570:
name: "default.Table_6a12fa2b84814dcbb515f51589cbf401"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:45.597868 11335 master_service.cc:946] ReplaceTablet: received request to replace tablet 9c766aac24e54c75a0e6202245a63234 from {username='slave'} at 127.0.0.1:40570
I20260808 13:36:45.597584 11334 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40570:
name: "default.Table_face27da876543c6b6151d51b3eef9e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260808 13:36:45.605644 11424 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_82a5f186cbcd4cf0875234550bb2665e
*** Check failure stack trace: ***
I20260808 13:36:45.606148 11333 master_service.cc:946] ReplaceTablet: received request to replace tablet 9c766aac24e54c75a0e6202245a63234 from {username='slave'} at 127.0.0.1:40570
I20260808 13:36:45.609059 11332 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:40570:
table { table_name: "default.Table_f7b46abc59bd4cacacf793d1d5a184b6" } new_table_name: "default.Table_7a5955f4f27e416a846a78664eb3987a" modify_external_catalogs: true
I20260808 13:36:45.611536 11321 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40570:
name: "default.Table_328786206e564026aa1c27ae85cf90f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:45.613611 11320 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40570:
name: "default.Table_e071104baa0d4c3baca39e9ac21c484f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:36:45.615643 11322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40570:
table { table_name: "default.Table_575a74e193bc45fd99dcd06ae9318532" } modify_external_catalogs: true
I20260808 13:36:45.616691 11322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40570:
table { table_name: "default.Table_575a74e193bc45fd99dcd06ae9318532" } modify_external_catalogs: true
@ 0x7f96fda3b9a0 google::LogMessage::Flush() at ??:0
I20260808 13:36:45.639699 11791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf9978620f543d8aee6279b9a9d3534. Latest consensus state: current_term: 6 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:45.640419 11791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:45.642132 11817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:45.642897 11817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:45.646840 4776 master-stress-test.cc:578] Killing test thread 1/10
I20260808 13:36:45.660643 11802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:45.661571 11802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534 [sys.catalog]: This master's current role is: LEADER
@ 0x7f96fda3b70f google::LogMessage::~LogMessage() at ??:0
I20260808 13:36:45.667124 11817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "0bf9978620f543d8aee6279b9a9d3534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } } }
I20260808 13:36:45.667819 11817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:36:45.675347 11335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bf9978620f543d8aee6279b9a9d3534: Sending DeleteTablet for 3 replicas of tablet 9c766aac24e54c75a0e6202245a63234
I20260808 13:36:45.684368 11814 mvcc.cc:205] Tried to move back new op lower bound from 7316259658013687808 to 7316259628969209856. Current Snapshot: MvccSnapshot[applied={T|T < 7316259658013687808 or (T in {7316259658013687808})}]
I20260808 13:36:45.708179 11335 catalog_manager.cc:6565] ReplaceTablet: tablet 9c766aac24e54c75a0e6202245a63234 deleted and replaced by tablet e47b64941d784cacaedc00c5cf8e2f25
I20260808 13:36:45.745121 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c766aac24e54c75a0e6202245a63234 with delete_type TABLET_DATA_DELETED (replaced by tablet e47b64941d784cacaedc00c5cf8e2f25) from {username='slave'} at 127.0.0.1:49298
I20260808 13:36:45.747484 10626 tablet_service.cc:1467] Processing CreateTablet for tablet e47b64941d784cacaedc00c5cf8e2f25 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:45.749006 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47b64941d784cacaedc00c5cf8e2f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:45.753334 11841 tablet_replica.cc:330] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995: stopping tablet replica
@ 0x7f9708ad6aea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260808 13:36:45.763980 11841 raft_consensus.cc:2229] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:45.764565 11841 raft_consensus.cc:2258] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:45.766897 11841 ts_tablet_manager.cc:1905] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:45.797899 10396 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c766aac24e54c75a0e6202245a63234 with delete_type TABLET_DATA_DELETED (replaced by tablet e47b64941d784cacaedc00c5cf8e2f25) from {username='slave'} at 127.0.0.1:56680
I20260808 13:36:45.799372 10393 tablet_service.cc:1467] Processing CreateTablet for tablet e47b64941d784cacaedc00c5cf8e2f25 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:45.800671 10393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47b64941d784cacaedc00c5cf8e2f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:45.806188 10866 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c766aac24e54c75a0e6202245a63234 with delete_type TABLET_DATA_DELETED (replaced by tablet e47b64941d784cacaedc00c5cf8e2f25) from {username='slave'} at 127.0.0.1:47900
I20260808 13:36:45.807641 10865 tablet_service.cc:1467] Processing CreateTablet for tablet e47b64941d784cacaedc00c5cf8e2f25 (DEFAULT_TABLE table=default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:36:45.809666 10865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47b64941d784cacaedc00c5cf8e2f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:36:45.819725 11843 tablet_bootstrap.cc:492] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995: Bootstrap starting.
I20260808 13:36:45.822194 11847 tablet_replica.cc:330] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37: stopping tablet replica
@ 0x7f9708ad633c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260808 13:36:45.824178 11845 tablet_replica.cc:330] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6: stopping tablet replica
I20260808 13:36:45.825075 11845 raft_consensus.cc:2229] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:36:45.825591 11845 raft_consensus.cc:2258] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:45.826836 11843 tablet_bootstrap.cc:654] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:45.827584 11847 raft_consensus.cc:2229] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:36:45.828900 11847 raft_consensus.cc:2258] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:36:45.831563 11847 ts_tablet_manager.cc:1905] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:45.836980 11845 ts_tablet_manager.cc:1905] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:36:45.852059 11841 ts_tablet_manager.cc:1918] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:45.852514 11841 log.cc:1196] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-1/wal/wals/9c766aac24e54c75a0e6202245a63234
I20260808 13:36:45.856606 11841 ts_tablet_manager.cc:1939] T 9c766aac24e54c75a0e6202245a63234 P e9d3df921f204d8aa02da52aca33d995: Deleting consensus metadata
I20260808 13:36:45.859596 11843 tablet_bootstrap.cc:492] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995: No bootstrap required, opened a new log
I20260808 13:36:45.860069 11843 ts_tablet_manager.cc:1397] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.006s
I20260808 13:36:45.864254 11845 ts_tablet_manager.cc:1918] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:45.864305 11847 ts_tablet_manager.cc:1918] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260808 13:36:45.864113 11843 raft_consensus.cc:348] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:45.864665 11845 log.cc:1196] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-0/wal/wals/9c766aac24e54c75a0e6202245a63234
I20260808 13:36:45.864939 11847 log.cc:1196] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting WAL directory at /tmp/dist-test-taskoVZaNS/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786196080693427-4776-0/minicluster-data/ts-2/wal/wals/9c766aac24e54c75a0e6202245a63234
I20260808 13:36:45.864941 11843 raft_consensus.cc:374] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:45.865760 11845 ts_tablet_manager.cc:1939] T 9c766aac24e54c75a0e6202245a63234 P 117d8c07fa254197969de761dd678aa6: Deleting consensus metadata
I20260808 13:36:45.865769 11843 raft_consensus.cc:729] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3df921f204d8aa02da52aca33d995, State: Initialized, Role: FOLLOWER
I20260808 13:36:45.866546 11843 consensus_queue.cc:260] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:45.870162 11854 tablet_bootstrap.cc:492] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6: Bootstrap starting.
I20260808 13:36:45.866341 11847 ts_tablet_manager.cc:1939] T 9c766aac24e54c75a0e6202245a63234 P e0e40695b6c9401aa0c178f58d8e2f37: Deleting consensus metadata
I20260808 13:36:45.872458 11849 tablet_bootstrap.cc:492] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37: Bootstrap starting.
I20260808 13:36:45.872812 11311 catalog_manager.cc:4841] TS e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): tablet 9c766aac24e54c75a0e6202245a63234 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:45.873728 11314 catalog_manager.cc:4841] TS 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): tablet 9c766aac24e54c75a0e6202245a63234 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:45.875695 10730 heartbeater.cc:503] Master 127.4.170.60:45415 was elected leader, sending a full tablet report...
I20260808 13:36:45.877665 11854 tablet_bootstrap.cc:654] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6: Neither blocks nor log segments found. Creating new log.
W20260808 13:36:45.882720 11333 catalog_manager.cc:5328] Ignoring report from unknown tablet 7426fee1d9dd43b7b02c73e683135c90
I20260808 13:36:45.883932 11843 ts_tablet_manager.cc:1428] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995: Time spent starting tablet: real 0.024s user 0.004s sys 0.003s
I20260808 13:36:45.890426 11314 catalog_manager.cc:4841] TS e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889): tablet 9c766aac24e54c75a0e6202245a63234 (table default.table_575a74e193bc45fd99dcd06ae9318532 [id=aaf27748800049ed98dbb95f79b7af14]) successfully deleted
I20260808 13:36:45.885690 11333 catalog_manager.cc:5495] T 662cbf673bf94bde9026d19cb51b9193 P e9d3df921f204d8aa02da52aca33d995 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3). New cstate: current_term: 1 leader_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } } }
I20260808 13:36:45.895465 11854 tablet_bootstrap.cc:492] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6: No bootstrap required, opened a new log
I20260808 13:36:45.895913 11854 ts_tablet_manager.cc:1397] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.016s
I20260808 13:36:45.896958 11849 tablet_bootstrap.cc:654] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37: Neither blocks nor log segments found. Creating new log.
I20260808 13:36:45.898422 11854 raft_consensus.cc:348] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:45.899165 11854 raft_consensus.cc:374] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:45.899634 11854 raft_consensus.cc:729] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Initialized, Role: FOLLOWER
I20260808 13:36:45.900293 11854 consensus_queue.cc:260] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:45.909224 11849 tablet_bootstrap.cc:492] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37: No bootstrap required, opened a new log
I20260808 13:36:45.909806 11849 ts_tablet_manager.cc:1397] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent bootstrapping tablet: real 0.038s user 0.019s sys 0.000s
I20260808 13:36:45.913086 11849 raft_consensus.cc:348] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
@ 0x7f9708ad9fd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260808 13:36:45.915403 10492 heartbeater.cc:503] Master 127.4.170.60:45415 was elected leader, sending a full tablet report...
I20260808 13:36:45.915788 11854 ts_tablet_manager.cc:1428] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6: Time spent starting tablet: real 0.020s user 0.007s sys 0.013s
I20260808 13:36:45.925666 11849 raft_consensus.cc:374] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:36:45.926030 11849 raft_consensus.cc:729] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e40695b6c9401aa0c178f58d8e2f37, State: Initialized, Role: FOLLOWER
I20260808 13:36:45.927580 11849 consensus_queue.cc:260] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:45.932579 10987 heartbeater.cc:503] Master 127.4.170.60:45415 was elected leader, sending a full tablet report...
I20260808 13:36:45.933190 11849 ts_tablet_manager.cc:1428] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37: Time spent starting tablet: real 0.023s user 0.002s sys 0.007s
W20260808 13:36:45.941267 11333 catalog_manager.cc:5328] Ignoring report from unknown tablet 9494b956367643ca9f4372a94bd84cc5
W20260808 13:36:45.941568 11333 catalog_manager.cc:5328] Ignoring report from unknown tablet 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:36:45.941913 11333 catalog_manager.cc:5328] Ignoring report from unknown tablet 7426fee1d9dd43b7b02c73e683135c90
@ 0x7f9708ad9f8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
@ 0x7f9708ad9f1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
@ 0x7f9708ad9ee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
@ 0x7f9708ad91de std::__1::__function::__func<>::operator()() at ??:0
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5805ba std::__1::function<>::operator()() at ??:?
@ 0x7f96ffbecdc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
@ 0x7f970125a6db start_thread at ??:0
@ 0x7f96fc6e061f clone at ??:0
W20260808 13:36:46.251911 11009 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.170.60:45415)
W20260808 13:36:46.252185 10793 connection.cc:535] client connection to 127.4.170.60:45415 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 13:36:46.253701 11009 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.4.170.60:45415)
W20260808 13:36:46.254822 11009 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.4.170.60:45415)
I20260808 13:36:46.304450 11859 raft_consensus.cc:482] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:46.304872 11859 raft_consensus.cc:504] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:46.306679 11859 leader_election.cc:288] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:46.307577 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47b64941d784cacaedc00c5cf8e2f25" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995" is_pre_election: true
I20260808 13:36:46.308172 10654 raft_consensus.cc:2454] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 0.
I20260808 13:36:46.308137 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47b64941d784cacaedc00c5cf8e2f25" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" is_pre_election: true
I20260808 13:36:46.308961 10905 raft_consensus.cc:2454] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 0.
I20260808 13:36:46.309530 10313 leader_election.cc:302] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:46.310276 11859 raft_consensus.cc:2781] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:36:46.310659 11859 raft_consensus.cc:482] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:36:46.310962 11859 raft_consensus.cc:3037] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:46.316710 11859 raft_consensus.cc:504] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:46.318172 11859 leader_election.cc:288] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 election: Requested vote from peers e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841), e0e40695b6c9401aa0c178f58d8e2f37 (127.4.170.3:35889)
I20260808 13:36:46.319190 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47b64941d784cacaedc00c5cf8e2f25" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3df921f204d8aa02da52aca33d995"
I20260808 13:36:46.319300 10905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47b64941d784cacaedc00c5cf8e2f25" candidate_uuid: "117d8c07fa254197969de761dd678aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e40695b6c9401aa0c178f58d8e2f37"
I20260808 13:36:46.319782 10654 raft_consensus.cc:3037] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:46.319847 10905 raft_consensus.cc:3037] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:36:46.326584 10654 raft_consensus.cc:2454] T e47b64941d784cacaedc00c5cf8e2f25 P e9d3df921f204d8aa02da52aca33d995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 1.
I20260808 13:36:46.326584 10905 raft_consensus.cc:2454] T e47b64941d784cacaedc00c5cf8e2f25 P e0e40695b6c9401aa0c178f58d8e2f37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d8c07fa254197969de761dd678aa6 in term 1.
I20260808 13:36:46.327562 10313 leader_election.cc:302] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995; no voters:
I20260808 13:36:46.328608 11859 raft_consensus.cc:2781] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:36:46.329465 11859 raft_consensus.cc:686] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [term 1 LEADER]: Becoming Leader. State: Replica: 117d8c07fa254197969de761dd678aa6, State: Running, Role: LEADER
I20260808 13:36:46.330273 11859 consensus_queue.cc:237] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:46.517169 11890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:46.517566 11890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:46.519178 11890 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:46.523801 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:36:46.524411 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:46.524713 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:46.525225 11890 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:36:47.126480 11902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:47.126965 11902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:47.128582 11902 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:47.137701 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:36:47.138116 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:36:47.138549 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:47.139165 11902 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:36:47.143029 11908 consensus_queue.cc:1035] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:36:47.162981 11916 consensus_queue.cc:1035] T e47b64941d784cacaedc00c5cf8e2f25 P 117d8c07fa254197969de761dd678aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 13:36:47.980381 11936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:47.980854 11936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:47.982366 11936 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:47.987807 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:36:47.988248 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:47.988664 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:47.989354 11936 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:36:48.078398 11951 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:48.078933 11951 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:48.080588 11951 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:36:48.082085 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:36:48.082545 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:36:48.082914 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:36:48.083428 11951 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:48.509176 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:36:48.509582 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:36:49.357764 11985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:49.358283 11985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:49.360014 11985 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:49.361927 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20260808 13:36:49.365054 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:36:49.365506 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:49.365885 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:49.366449 11985 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:36:49.971179 12013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:49.971616 12013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:49.973489 12013 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:49.982256 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:36:49.983397 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:36:49.983814 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:49.984542 12013 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:36:50.921185 12043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:50.921636 12043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:50.923381 12043 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:50.928105 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:36:50.928537 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:50.928875 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:50.929410 12043 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:36:51.757092 12067 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:51.757774 12067 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:51.760289 12067 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:36:51.762261 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:36:51.762943 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:36:51.763306 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:36:51.764034 12067 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:36:53.513121 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:36:53.513599 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:36:53.648561 12095 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:36:53.649013 12095 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:36:53.650655 12095 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:36:53.651993 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:36:53.652439 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:36:53.652776 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:36:53.653304 12095 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:36:54.785365 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 11: this message will repeat every 5th retry.
I20260808 13:36:56.597030 12173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:56.597623 12173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:56.599790 12173 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:56.602061 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260808 13:36:56.605206 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:36:56.605844 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:56.606251 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:56.607095 12173 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:36:58.516211 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:36:58.516584 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:36:59.322742 12248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:36:59.323141 12248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:36:59.324756 12248 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:36:59.329371 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:36:59.329864 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:36:59.330211 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:36:59.330818 12248 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:03.350454 12358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:03.351037 12358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:37:03.352721 12358 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:03.355048 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260808 13:37:03.358194 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:37:03.358919 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:37:03.359207 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:03.359791 12358 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:03.519203 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:03.519665 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:37:03.772570 10485 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3ce0 after lost signal to thread 10277
W20260808 13:37:03.773765 10485 debug-util.cc:398] Leaking SignalData structure 0x7b08000d90c0 after lost signal to thread 10490
W20260808 13:37:03.896194 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 16: this message will repeat every 5th retry.
I20260808 13:37:04.338536 12376 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:04.339282 12376 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:37:04.342161 12376 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:37:04.343502 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:37:04.344089 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:37:04.344516 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:04.345347 12376 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:37:04.448280 12376 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:04.448747 12376 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:37:04.450435 12376 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:37:04.451960 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:37:04.452417 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:37:04.452922 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:04.453831 12376 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:37:06.977057 11721 debug-util.cc:398] Leaking SignalData structure 0x7b08000af660 after lost signal to thread 11592
W20260808 13:37:06.977794 11721 debug-util.cc:398] Leaking SignalData structure 0x7b0800098760 after lost signal to thread 11726
I20260808 13:37:08.565552 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:08.566018 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:37:08.842859 12483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:08.843286 12483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:37:08.844894 12483 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:08.846395 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:37:08.849931 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:37:08.850471 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:37:08.850912 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:08.851588 12483 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:09.583027 12506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:09.583465 12506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:37:09.585215 12506 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:09.591259 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:37:09.591779 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:37:09.592321 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:09.593211 12506 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:11.405928 12494 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:11.406642 12494 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:37:11.409335 12494 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:37:11.411341 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:37:11.411887 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:37:11.412247 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:11.413012 12494 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:37:13.569864 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:13.570314 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:37:14.230250 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 21: this message will repeat every 5th retry.
I20260808 13:37:16.111665 12652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:16.112169 12652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:37:16.113857 12652 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:16.115896 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260808 13:37:16.118904 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:37:16.119346 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:37:16.119669 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:16.120186 12652 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:17.857751 12695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:17.858224 12695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:37:17.860028 12695 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:17.865432 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:37:17.865862 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:37:17.866261 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:17.866940 12695 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:18.573333 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:18.573675 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:37:19.422760 12741 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:19.423418 12741 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:37:19.427757 12741 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:37:19.429097 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:37:19.429718 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:37:19.430058 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:19.430691 12741 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:37:21.549367 10723 debug-util.cc:398] Leaking SignalData structure 0x7b08000438e0 after lost signal to thread 10523
W20260808 13:37:21.551919 10723 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd8a0 after lost signal to thread 10729
I20260808 13:37:21.913520 12777 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:21.914507 12777 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
W20260808 13:37:21.929337 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:37:21.946749 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:37:21.961593 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:21.977015 12777 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
I20260808 13:37:21.984200 12777 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:37:22.020581 10791 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.4.170.2:34841, user_credentials={real_user=slave}} blocked reactor thread for 74776.1us
I20260808 13:37:22.319027 12792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:22.323089 12792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
W20260808 13:37:22.395716 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 3 similar messages]
I20260808 13:37:22.415774 12792 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:22.424782 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:37:22.442181 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:37:22.443687 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:22.446323 12792 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:23.576865 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:23.577533 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:37:24.153874 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 26: this message will repeat every 5th retry.
I20260808 13:37:28.580394 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:28.580943 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:37:31.825538 11721 debug-util.cc:398] Leaking SignalData structure 0x7b08000c3740 after lost signal to thread 11592
W20260808 13:37:31.826471 11721 debug-util.cc:398] Leaking SignalData structure 0x7b080003e600 after lost signal to thread 11726
I20260808 13:37:31.948316 12911 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:31.949044 12911 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:37:31.952772 12911 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:37:31.954386 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:37:31.955134 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:37:31.955508 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:31.956339 12911 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:37:32.400444 12922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:32.401063 12922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:37:32.403131 12922 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:32.405805 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:37:32.409415 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:37:32.409947 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:37:32.410274 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:32.411087 12922 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260808 13:37:33.408453 10496 heartbeater.cc:644] Failed to heartbeat to 127.4.170.62:46707 (112 consecutive failures): Network error: Failed to ping master at 127.4.170.62:46707: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:37:33.557787 10987 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (117 consecutive failures): Network error: Failed to ping master at 127.4.170.60:45415: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:37:33.582258 10730 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (95 consecutive failures): Network error: Failed to ping master at 127.4.170.60:45415: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:37:33.599453 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:33.599879 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:37:34.211983 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 31: this message will repeat every 5th retry.
I20260808 13:37:36.569334 12973 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:36.569753 12973 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:37:36.571512 12973 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:37:36.573345 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:37:36.573750 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:37:36.574021 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:36.574554 12973 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:37:38.384510 12998 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:38.384974 12998 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:37:38.386740 12998 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:37:38.388123 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:37:38.388520 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:37:38.388814 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:38.389394 12998 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:37:38.602587 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:38.603155 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:37:41.273980 13037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:41.274389 13037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:37:41.276050 13037 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:41.277918 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:37:41.280704 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:37:41.281257 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:37:41.281625 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:41.282208 13037 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:43.454027 13058 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:43.454758 13058 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:37:43.456810 13058 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:37:43.458433 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:37:43.458990 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:37:43.459266 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:43.459915 13058 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:37:43.606415 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:43.606935 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:37:44.032814 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 36: this message will repeat every 5th retry.
I20260808 13:37:44.044051 13070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:44.044512 13070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:37:44.046190 13070 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:44.050804 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:37:44.051471 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:37:44.051771 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:44.052361 13070 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:45.832633 13095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:45.833173 13095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:37:45.834931 13095 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:45.839447 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:37:45.839982 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:37:45.840376 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:45.840970 13095 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:48.199090 13126 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:48.199541 13126 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:37:48.201385 13126 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:37:48.203166 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:37:48.203540 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:37:48.203819 10796 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:48.204350 13126 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:37:48.609764 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:48.610154 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:37:52.213261 13163 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:37:52.213976 13163 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:37:52.216917 13163 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:37:52.217830 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:37:52.218429 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:37:52.218726 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:37:52.219331 13163 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:37:52.524421 13169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:37:52.525008 13169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:37:52.526791 13169 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:37:52.529284 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260808 13:37:52.532264 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:37:52.532673 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:37:52.532992 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:37:52.533706 13169 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:37:53.613530 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:53.614079 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:37:54.288841 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 41: this message will repeat every 5th retry.
I20260808 13:37:58.616868 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:37:58.617273 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:38:00.404268 10982 debug-util.cc:398] Leaking SignalData structure 0x7b08000e9040 after lost signal to thread 10747
W20260808 13:38:00.405149 10982 debug-util.cc:398] Leaking SignalData structure 0x7b08000da580 after lost signal to thread 10986
I20260808 13:38:03.290711 13289 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:03.291746 13289 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:03.294924 13289 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:03.296656 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:38:03.297434 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:38:03.297683 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:03.298178 13289 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:38:03.620519 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:03.620978 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:38:04.284281 10982 debug-util.cc:398] Leaking SignalData structure 0x7b08000c8e40 after lost signal to thread 10747
W20260808 13:38:04.285228 10982 debug-util.cc:398] Leaking SignalData structure 0x7b08000438a0 after lost signal to thread 10986
W20260808 13:38:04.370003 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 46: this message will repeat every 5th retry.
I20260808 13:38:07.688319 13345 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:07.688768 13345 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:07.690496 13345 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:07.692440 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:38:07.692909 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:38:07.693204 10796 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:07.693835 13345 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:38:08.624351 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:08.624823 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:38:11.740969 13419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:38:11.741425 13419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:38:11.743593 13419 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:38:11.746048 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:38:11.749595 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:38:11.750324 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:38:11.750694 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:38:11.751313 13419 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:38:13.628109 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:13.628530 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:38:14.095647 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 51: this message will repeat every 5th retry.
I20260808 13:38:14.927209 13467 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:14.927909 13467 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:14.930320 13467 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:14.931857 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:38:14.932410 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:38:14.932657 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:14.933228 13467 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:38:18.631345 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:18.631726 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:38:22.833741 13609 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:22.834316 13609 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:22.836694 13609 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:22.838351 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:38:22.838876 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:38:22.839188 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:22.839746 13609 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:38:23.635231 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:23.635708 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:38:24.124598 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 56: this message will repeat every 5th retry.
I20260808 13:38:24.614790 13643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:38:24.615283 13643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:38:24.616904 13643 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:38:24.619387 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:38:24.622287 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:38:24.622834 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:38:24.623442 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:38:24.624058 13643 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:38:28.638387 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:28.638845 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:38:31.319360 13750 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:31.320436 13750 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:31.324884 13750 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:31.325945 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:38:31.326678 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:38:31.327152 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:31.327862 13750 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:38:33.517283 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 61: this message will repeat every 5th retry.
I20260808 13:38:33.642261 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:33.642808 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:38:33.660225 10496 heartbeater.cc:644] Failed to heartbeat to 127.4.170.62:46707 (232 consecutive failures): Network error: Failed to ping master at 127.4.170.62:46707: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:38:33.915123 10730 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (215 consecutive failures): Network error: Failed to ping master at 127.4.170.60:45415: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:38:34.021945 10987 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (270 consecutive failures): Network error: Failed to ping master at 127.4.170.60:45415: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:38:35.467614 13837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:38:35.468078 13837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:38:35.469744 13837 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:38:35.471880 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:38:35.475073 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:38:35.475484 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:38:35.475831 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:38:35.476383 13837 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:38:37.052377 13850 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:37.052970 13850 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:37.054913 13850 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:37.056571 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:38:37.056958 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:38:37.057216 10796 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:37.057734 13850 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:38:38.645560 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:38.646015 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:38:38.681236 13888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:38:38.681684 13888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:38:38.683537 13888 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:38:38.687005 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:38:38.687408 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:38:38.687716 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:38:38.688298 13888 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:38:41.392899 13925 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:41.393695 13925 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:41.396514 13925 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:41.398470 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:38:41.399045 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:38:41.399369 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:41.399998 13925 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:38:41.566597 13925 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:41.567119 13925 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:41.568714 13925 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:41.570345 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:38:41.570777 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:38:41.571095 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:41.571636 13925 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:38:43.520335 11652 maintenance_manager.cc:419] P 026dcc55447e4441b062e327f64eacb4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033356
I20260808 13:38:43.603825 11651 maintenance_manager.cc:643] P 026dcc55447e4441b062e327f64eacb4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.082s user 0.044s sys 0.018s Metrics: {"bytes_written":12825,"cfile_init":1,"drs_written":1,"lbm_read_time_us":215,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2591,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":23,"thread_start_us":430,"threads_started":1,"wal-append.queue_time_us":735}
I20260808 13:38:43.605993 11652 maintenance_manager.cc:419] P 026dcc55447e4441b062e327f64eacb4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1278 bytes on disk
I20260808 13:38:43.607586 11651 maintenance_manager.cc:643] P 026dcc55447e4441b062e327f64eacb4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":234,"lbm_reads_lt_1ms":4}
I20260808 13:38:43.649719 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:43.650125 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:38:43.805037 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 66: this message will repeat every 5th retry.
I20260808 13:38:48.653079 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:48.653450 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:38:53.656865 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:53.657387 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:38:53.942955 14126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:38:53.943647 14126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:38:53.946458 14126 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:38:53.949025 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260808 13:38:53.954434 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:38:53.955185 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:38:53.955560 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:38:53.956471 14126 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260808 13:38:54.059693 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 71: this message will repeat every 5th retry.
I20260808 13:38:54.519497 14136 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:54.520085 14136 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:54.522105 14136 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:54.524101 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:38:54.524479 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:38:54.524734 10796 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:54.525259 14136 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:38:55.424662 14152 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:55.425170 14152 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:55.427055 14152 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:55.428635 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:38:55.429046 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:38:55.429359 10796 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:55.429927 14152 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:38:58.282820 10982 debug-util.cc:398] Leaking SignalData structure 0x7b08000e6960 after lost signal to thread 10747
W20260808 13:38:58.283663 10982 debug-util.cc:398] Leaking SignalData structure 0x7b08000ec940 after lost signal to thread 10986
I20260808 13:38:58.661031 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:38:58.661520 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:38:59.855463 14232 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:38:59.856099 14232 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:38:59.864454 14232 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:38:59.866132 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:38:59.874148 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:38:59.874630 10796 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:38:59.875396 14232 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:39:00.876559 10485 debug-util.cc:398] Leaking SignalData structure 0x7b08000dda60 after lost signal to thread 10277
W20260808 13:39:00.877478 10485 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5ca0 after lost signal to thread 10490
W20260808 13:39:01.319772 11721 debug-util.cc:398] Leaking SignalData structure 0x7b08000a0520 after lost signal to thread 11592
W20260808 13:39:01.320367 11721 debug-util.cc:398] Leaking SignalData structure 0x7b08000a8580 after lost signal to thread 11726
I20260808 13:39:01.491683 14291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:39:01.492147 14291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:39:01.493958 14291 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:39:01.496089 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:39:01.498548 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:39:01.499395 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:39:01.499678 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:39:01.500299 14291 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:39:03.704201 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:03.704696 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:39:04.416464 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 76: this message will repeat every 5th retry.
I20260808 13:39:08.707504 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:08.707906 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:39:13.460314 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 81: this message will repeat every 5th retry.
I20260808 13:39:13.711563 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:13.712098 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:39:14.587381 14574 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:39:14.587930 14574 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:39:14.589844 14574 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:39:14.591768 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:39:14.592357 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:39:14.592705 10796 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:39:14.593425 14574 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:39:15.007385 10723 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5f00 after lost signal to thread 10523
W20260808 13:39:15.008380 10723 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3f40 after lost signal to thread 10729
I20260808 13:39:15.496980 14621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:39:15.497545 14621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:39:15.499449 14621 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:39:15.501516 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:39:15.504442 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:39:15.504987 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:39:15.505337 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:39:15.505918 14621 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:39:18.143038 14679 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:39:18.143512 14679 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:39:18.145679 14679 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:39:18.147392 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:39:18.147816 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:39:18.148140 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:39:18.148767 14679 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:39:18.715224 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:18.715611 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:39:22.197783 14791 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:39:22.198421 14791 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:39:22.200461 14791 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:39:22.201972 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:39:22.202560 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:39:22.202898 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:39:22.203478 14791 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:39:23.664840 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 86: this message will repeat every 5th retry.
I20260808 13:39:23.718917 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:23.719399 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:39:28.404062 14947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:39:28.404577 14947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:39:28.406291 14947 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:39:28.408264 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:39:28.411273 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:39:28.411885 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:39:28.412253 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:39:28.412886 14947 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:39:28.722379 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:28.722983 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:39:33.101591 15046 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:39:33.102185 15046 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:39:33.104180 15046 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:39:33.105991 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:39:33.106523 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:39:33.106901 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:39:33.107553 15046 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:39:33.454928 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 91: this message will repeat every 5th retry.
I20260808 13:39:33.726822 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:33.727313 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:39:33.924823 10496 heartbeater.cc:644] Failed to heartbeat to 127.4.170.62:46707 (352 consecutive failures): Network error: Failed to ping master at 127.4.170.62:46707: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:39:34.306965 10730 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (335 consecutive failures): Network error: Failed to ping master at 127.4.170.60:45415: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:39:34.461547 10991 heartbeater.cc:644] Failed to heartbeat to 127.4.170.62:46707 (444 consecutive failures): Network error: Failed to ping master at 127.4.170.62:46707: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:39:34.838518 15092 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:39:34.839040 15092 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:39:34.841802 15092 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:39:34.842988 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:39:34.843384 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:39:34.843670 10796 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:39:34.844278 15092 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:39:38.729991 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:38.730428 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:39:42.882834 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 96: this message will repeat every 5th retry.
I20260808 13:39:43.733995 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:43.734525 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:39:47.398399 15383 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:39:47.399152 15383 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:39:47.401263 15383 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:39:47.402957 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:39:47.403553 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:39:47.403852 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:39:47.404448 15383 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:39:47.645468 15389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:39:47.646037 15389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:39:47.648655 15389 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:39:47.650866 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:39:47.653931 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:39:47.654663 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:39:47.654992 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:39:47.655580 15389 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:39:48.737536 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:48.737964 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:39:49.195825 15422 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:39:49.196295 15422 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:39:49.197942 15422 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:39:49.199493 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:39:49.199925 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:39:49.200188 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:39:49.200678 15422 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:39:53.637059 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 101: this message will repeat every 5th retry.
I20260808 13:39:53.741472 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:53.742004 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:39:56.635506 15551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:39:56.636025 15551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:39:56.637802 15551 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:39:56.640246 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:39:56.643164 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:39:56.643759 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:39:56.644033 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:39:56.644673 15551 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:39:58.745723 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:39:58.746258 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:39:58.905090 15583 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:39:58.905681 15583 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:39:58.908061 15583 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:39:58.909627 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:39:58.910118 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:39:58.910440 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:39:58.911020 15583 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:39:59.737185 10485 debug-util.cc:398] Leaking SignalData structure 0x7b08000bbf80 after lost signal to thread 10277
W20260808 13:39:59.738099 10485 debug-util.cc:398] Leaking SignalData structure 0x7b08000b73c0 after lost signal to thread 10490
W20260808 13:40:03.033577 10723 debug-util.cc:398] Leaking SignalData structure 0x7b08000705c0 after lost signal to thread 10523
W20260808 13:40:03.034710 10723 debug-util.cc:398] Leaking SignalData structure 0x7b0800070460 after lost signal to thread 10729
I20260808 13:40:03.750589 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:03.751118 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:40:03.813122 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 106: this message will repeat every 5th retry.
I20260808 13:40:05.434504 15675 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:40:05.435104 15675 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:40:05.437049 15675 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:40:05.438637 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:40:05.439247 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:40:05.439603 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:40:05.440168 15675 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:40:08.754046 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:08.754494 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:40:12.924819 15835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:40:12.925414 15835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:40:12.927788 15835 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:40:12.930125 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:40:12.932929 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:40:12.933446 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:40:12.933712 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:40:12.934268 15835 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:40:13.069480 15833 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:40:13.070104 15833 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:40:13.072232 15833 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:40:13.074082 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:40:13.074792 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:40:13.075093 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:40:13.075659 15833 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:40:13.757850 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:13.758435 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:40:14.720911 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 111: this message will repeat every 5th retry.
I20260808 13:40:18.761464 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:18.761948 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:40:18.947639 15926 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:40:18.948125 15926 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:40:18.950173 15926 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:40:18.951799 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:40:18.952226 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:40:18.952513 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:40:18.953083 15926 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:40:23.765864 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:23.766458 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:40:23.929950 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 116: this message will repeat every 5th retry.
W20260808 13:40:26.683483 10485 debug-util.cc:398] Leaking SignalData structure 0x7b08000d32a0 after lost signal to thread 10277
W20260808 13:40:26.684417 10485 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5d20 after lost signal to thread 10490
I20260808 13:40:27.980317 16074 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:40:27.980919 16074 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:40:27.982846 16074 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:40:27.984512 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:40:27.984997 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:40:27.985283 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:40:27.985872 16074 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:40:28.769685 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:28.770109 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:40:28.842834 16101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:40:28.843369 16101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:40:28.845542 16101 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:40:28.847218 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:40:28.850103 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:40:28.850648 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:40:28.850956 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:40:28.851560 16101 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260808 13:40:31.965657 10723 debug-util.cc:398] Leaking SignalData structure 0x7b0800070400 after lost signal to thread 10523
W20260808 13:40:31.966776 10723 debug-util.cc:398] Leaking SignalData structure 0x7b0800070860 after lost signal to thread 10729
I20260808 13:40:32.750511 16176 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:40:32.751262 16176 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:40:32.753472 16176 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:40:32.754979 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:40:32.755540 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:40:32.755860 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:40:32.756440 16176 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:40:33.358034 16199 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:40:33.358475 16199 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:40:33.360602 16199 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:40:33.361663 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:40:33.362037 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:40:33.362396 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:40:33.363016 16199 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:40:33.773994 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:33.774487 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:40:34.224881 10496 heartbeater.cc:644] Failed to heartbeat to 127.4.170.62:46707 (472 consecutive failures): Network error: Failed to ping master at 127.4.170.62:46707: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:40:34.471881 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 121: this message will repeat every 5th retry.
W20260808 13:40:34.472203 10987 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (571 consecutive failures): Network error: Failed to ping master at 127.4.170.60:45415: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:40:34.673110 10732 heartbeater.cc:644] Failed to heartbeat to 127.4.170.62:46707 (472 consecutive failures): Network error: Failed to ping master at 127.4.170.62:46707: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:40:34.731513 16233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:40:34.732082 16233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:40:34.733968 16233 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:40:34.736006 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:40:34.738894 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:40:34.739517 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:40:34.739779 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:40:34.740403 16233 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:40:35.563642 16256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:40:35.564085 16256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:40:35.566097 16256 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:40:35.570546 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:40:35.570996 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:40:35.571324 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:40:35.571915 16256 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260808 13:40:37.193385 10982 debug-util.cc:398] Leaking SignalData structure 0x7b08000b5da0 after lost signal to thread 10747
W20260808 13:40:37.194315 10982 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7d80 after lost signal to thread 10986
W20260808 13:40:37.583025 10723 debug-util.cc:398] Leaking SignalData structure 0x7b08000709a0 after lost signal to thread 10523
W20260808 13:40:37.583904 10723 debug-util.cc:398] Leaking SignalData structure 0x7b08000de140 after lost signal to thread 10729
W20260808 13:40:38.662240 10723 debug-util.cc:398] Leaking SignalData structure 0x7b0800070ac0 after lost signal to thread 10523
W20260808 13:40:38.663163 10723 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6dc0 after lost signal to thread 10729
I20260808 13:40:38.820729 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:38.821166 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:40:39.409850 16308 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:40:39.410295 16308 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:40:39.412268 16308 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:40:39.414057 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:40:39.414455 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:40:39.414800 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:40:39.415463 16308 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 13:40:39.767625 11721 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9380 after lost signal to thread 11592
W20260808 13:40:39.768268 11721 debug-util.cc:398] Leaking SignalData structure 0x7b08000a8f40 after lost signal to thread 11726
I20260808 13:40:43.824986 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:43.825516 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:40:45.119019 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 126: this message will repeat every 5th retry.
I20260808 13:40:46.704991 16489 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:40:46.705489 16489 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:40:46.707207 16489 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:40:46.708688 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:40:46.709242 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:40:46.709511 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:40:46.710052 16489 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:40:48.828461 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:48.828944 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:40:49.822762 16565 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:40:49.823200 16565 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:40:49.824878 16565 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:40:49.826396 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:40:49.826843 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:40:49.827147 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:40:49.827841 16565 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:40:53.298519 16650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:40:53.299191 16650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:40:53.301366 16650 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:40:53.303867 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260808 13:40:53.306691 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:40:53.307143 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
I20260808 13:40:53.307451 11649 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:40:53.307994 16650 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:40:53.832074 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:53.832568 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:40:55.980597 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 131: this message will repeat every 5th retry.
I20260808 13:40:55.987458 16705 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:40:55.988001 16705 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:40:55.990585 16705 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:40:55.991936 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:40:55.992512 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:40:55.992756 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:40:55.993227 16705 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:40:58.835129 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:40:58.835546 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:41:03.247478 16863 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:41:03.248108 16863 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:41:03.249949 16863 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:41:03.251749 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:41:03.252197 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:41:03.252467 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:41:03.253010 16863 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:41:03.839279 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:41:03.839821 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:41:05.933054 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 136: this message will repeat every 5th retry.
I20260808 13:41:07.233474 16939 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:41:07.234030 16939 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:41:07.236004 16939 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:41:07.238142 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:41:07.239470 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:41:07.239830 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:41:07.240427 16939 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:41:08.842666 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:41:08.843098 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:41:09.354586 16985 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:41:09.355048 16985 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:41:09.356781 16985 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:41:09.358238 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:41:09.358875 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:41:09.359179 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:41:09.359709 16985 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:41:12.290949 17034 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:41:12.291630 17034 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:41:12.294055 17034 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:41:12.295859 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:41:12.296413 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:41:12.296664 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:41:12.297309 17034 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:41:12.921617 17062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:41:12.922224 17062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:41:12.924520 17062 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:41:12.926828 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:41:12.929626 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:41:12.930212 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:41:12.930471 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:41:12.931041 17062 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:41:13.845825 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:41:13.846267 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:41:15.397001 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 141: this message will repeat every 5th retry.
I20260808 13:41:17.806080 17144 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:41:17.806658 17144 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:41:17.808647 17144 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:41:17.810361 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:41:17.810827 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:41:17.811175 10796 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:41:17.811740 17144 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:41:18.849149 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:41:18.849576 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:41:21.552525 17230 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:41:21.552933 17230 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:41:21.554536 17230 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:41:21.555991 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:41:21.556398 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:41:21.556706 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:41:21.557240 17230 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:41:23.853261 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:41:23.853694 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:41:24.459698 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 146: this message will repeat every 5th retry.
I20260808 13:41:25.003657 17325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bf9978620f543d8aee6279b9a9d3534)
I20260808 13:41:25.004182 17325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2c297fb28f4950864f0487ef83e833" member_type: VOTER last_known_addr { host: "127.4.170.62" port: 46707 } } peers { permanent_uuid: "026dcc55447e4441b062e327f64eacb4" member_type: VOTER last_known_addr { host: "127.4.170.61" port: 37949 } } peers { permanent_uuid: "0bf9978620f543d8aee6279b9a9d3534" member_type: VOTER last_known_addr { host: "127.4.170.60" port: 45415 } }
I20260808 13:41:25.005934 17325 leader_election.cc:288] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707), 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415)
W20260808 13:41:25.008512 11649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260808 13:41:25.011628 11649 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bc2c297fb28f4950864f0487ef83e833 (127.4.170.62:46707): Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:41:25.012198 11647 leader_election.cc:334] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bf9978620f543d8aee6279b9a9d3534 (127.4.170.60:45415): Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
I20260808 13:41:25.012480 11647 leader_election.cc:302] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 026dcc55447e4441b062e327f64eacb4; no voters: 0bf9978620f543d8aee6279b9a9d3534, bc2c297fb28f4950864f0487ef83e833
I20260808 13:41:25.013060 17325 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 026dcc55447e4441b062e327f64eacb4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260808 13:41:28.864318 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:41:28.864734 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:41:30.010289 17435 raft_consensus.cc:482] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:41:30.010890 17435 raft_consensus.cc:504] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:41:30.013298 17435 leader_election.cc:288] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:41:30.015175 10796 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
W20260808 13:41:30.023393 10791 leader_election.cc:341] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 9494b956367643ca9f4372a94bd84cc5
I20260808 13:41:30.023713 10791 leader_election.cc:302] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:41:30.024331 17435 raft_consensus.cc:2726] T 9494b956367643ca9f4372a94bd84cc5 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:41:33.836766 17521 raft_consensus.cc:482] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:41:33.837402 17521 raft_consensus.cc:504] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117d8c07fa254197969de761dd678aa6" member_type: VOTER last_known_addr { host: "127.4.170.1" port: 35471 } } peers { permanent_uuid: "e9d3df921f204d8aa02da52aca33d995" member_type: VOTER last_known_addr { host: "127.4.170.2" port: 34841 } } peers { permanent_uuid: "e0e40695b6c9401aa0c178f58d8e2f37" member_type: VOTER last_known_addr { host: "127.4.170.3" port: 35889 } }
I20260808 13:41:33.839625 17521 leader_election.cc:288] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471), e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841)
W20260808 13:41:33.841385 10796 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
W20260808 13:41:33.841907 10791 leader_election.cc:341] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9d3df921f204d8aa02da52aca33d995 (127.4.170.2:34841): Not found: Tablet not found: 3e117c2bf59245a89dad2c6e96bdcfb6
I20260808 13:41:33.842197 10791 leader_election.cc:302] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0e40695b6c9401aa0c178f58d8e2f37; no voters: 117d8c07fa254197969de761dd678aa6, e9d3df921f204d8aa02da52aca33d995
I20260808 13:41:33.842800 17521 raft_consensus.cc:2726] T 3e117c2bf59245a89dad2c6e96bdcfb6 P e0e40695b6c9401aa0c178f58d8e2f37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:41:33.869596 11583 master_runner.cc:186] Error getting master registration for 127.4.170.62:46707: Network error: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111),
I20260808 13:41:33.870188 11583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:41:34.215245 10796 consensus_peers.cc:487] T 7426fee1d9dd43b7b02c73e683135c90 P e0e40695b6c9401aa0c178f58d8e2f37 -> Peer 117d8c07fa254197969de761dd678aa6 (127.4.170.1:35471): Couldn't send request to peer 117d8c07fa254197969de761dd678aa6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426fee1d9dd43b7b02c73e683135c90. This is attempt 151: this message will repeat every 5th retry.
W20260808 13:41:34.475492 10496 heartbeater.cc:644] Failed to heartbeat to 127.4.170.62:46707 (592 consecutive failures): Network error: Failed to ping master at 127.4.170.62:46707: Client connection negotiation failed: client connection to 127.4.170.62:46707: connect: Connection refused (error 111)
W20260808 13:41:34.718113 10987 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (721 consecutive failures): Network error: Failed to ping master at 127.4.170.60:45415: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:41:35.060302 10730 heartbeater.cc:644] Failed to heartbeat to 127.4.170.60:45415 (575 consecutive failures): Network error: Failed to ping master at 127.4.170.60:45415: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:41:35.172924 11005 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415, num_attempts: 754) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:41:35.173563 11005 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415, num_attempts: 754) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
W20260808 13:41:35.173877 11005 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415, num_attempts: 754) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
F20260808 13:41:35.186422 11029 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415, num_attempts: 754) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260808 13:41:35.189184 11026 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e071104baa0d4c3baca39e9ac21c484f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.4.170.62:46707,127.4.170.61:37949,127.4.170.60:45415, num_attempts: 754) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.4.170.60:45415: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1786196495 (unix time) try "date -d @1786196495" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e8000012a8) received by PID 4776 (TID 0x7f666d5f8700) from PID 4776; stack trace: ***
@ 0x2b0af0 __tsan::CallUserSignalHandler() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b31e0 rtl_sigaction() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7f6680978980 (unknown) at ??:0
@ 0x7f667bc35e87 gsignal at ??:0
@ 0x7f667bc377f1 abort at ??:0
@ 0x2b4f37 __interceptor_abort at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7f667ec69df4 google::LogMessage::SendToLog() at ??:0
@ 0x7f667ec6a9a0 google::LogMessage::Flush() at ??:0
@ 0x7f667ec6fadb google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x357ade kudu::MasterStressTest::DeleteTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x3874ce _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3840
@ 0x387381 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:281
@ 0x386acb std::__1::__thread_proxy<>() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2b11b0 __tsan_thread_start_func at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
@ 0x7f668096d6db start_thread at ??:0
@ 0x7f667bd1861f clone at ??:0